A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes

# A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes††thanks: \fundingThis work was supported in part by NSF grant 2229876, the A. Russell Chandler III Professorship at Georgia Institute of Technology, an NIH-sponsored Georgia Clinical & Translational Science Alliance, and the Georgia Department of Transportation.
Source: [https://arxiv.org/html/2608.10470](https://arxiv.org/html/2608.10470)
\\newsiamremark

remarkRemark\\newsiamthmassumptionAssumption\\headersJoint\-Distribution Route to Fair RepresentationsY\. Ni and X\. Huo

Yijin NiH\. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332\-0205 USA \(, \)\. ORCID iDs:[0000\-0002\-7209\-6887](https://orcid.org/0000-0002-7209-6887)\(Ni\) and[0000\-0003\-0101\-1206](https://orcid.org/0000-0003-0101-1206)\(Huo\)\.

###### Abstract

Fair representation learning with a continuous sensitive attributeSSrequires a representationZZthat is statistically independent ofSS\. Existing criteria, including generalized demographic parity, the expectation of integral probability metrics \(EIPM\), and mutual information, enforce this independence by averaging a per\-value discrepancy between the conditional lawPZ∣S=sP\_\{Z\\mid S=s\}and the marginalPZP\_\{Z\}over the law ofSS\. This approach requires a nonparametric surrogate for the conditional law at each sensitive value\. We propose evaluating independence through a single joint discrepancyd​\(PZ,S,PZ⊗PS\)d\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\)between the joint law and the product of its marginals\. We establish a disintegration identity; on decomposable witness classes it equals the conditional\-integral functional that EIPM and generalized demographic parity instantiate\. By reaching the same target without the conditional law, this discrepancy can be estimated directly from samples via a dependence statistic rather than conditional smoothing\. We take the Hilbert–Schmidt independence criterion \(HSIC\) as an instance of the joint discrepancyddto investigate the statistical efficiency of replacing the conditional formulation\. The HSIC estimator is a closed\-formO​\(n2\)O\(n^\{2\}\)statistic that converges at theO​\(n−1/2\)O\(n^\{\-1/2\}\)rate, in contrast to the nonparametricO​\(n−2/5\)O\(n^\{\-2/5\}\)rate of the conditional\-route estimators\. We prove this instance is equivalent to the conditional maximum mean discrepancy \(MMD\) integral up to an explicit spectral tail\. The corresponding algorithmic implementation, i\.e\., FRHSIC, attains fairness–accuracy tradeoffs comparable to conditional\-route baselines while reducing per\-epoch training time\.

###### keywords:

fair representation learning, continuous sensitive attributes, Hilbert–Schmidt independence criterion, kernel methods, demographic parity

\{MSCcodes\}

68T05, 62G05, 62G20, 46E22

## 1Introduction

Fair representation learning seeks a representationZZthat is statistically independent of a sensitive attributeS∈𝒮S\\in\\mathcal\{S\}, i\.e\.,Z⟂SZ\\perp S, so that any downstream predictor built onZZinherits demographic parity with respect toSS\(Zemel et al\.,[2013](https://arxiv.org/html/2608.10470#bib.bib38); Madras et al\.,[2018](https://arxiv.org/html/2608.10470#bib.bib24)\)\. An encoderh:𝒳→𝒵h:\\mathcal\{X\}\\to\\mathcal\{Z\}maps an inputX∈𝒳X\\in\\mathcal\{X\}to the representationZ=h​\(X\)∈𝒵Z=h\(X\)\\in\\mathcal\{Z\}, and any prediction head acting onZZis fair onceZZcarries no information aboutSS, which makesZ⟂SZ\\perp Sthe representation\-level target\. WritingPZP\_\{Z\}andPSP\_\{S\}for the marginal laws ofZZandSSandPZ∣S=sP\_\{Z\\mid S=s\}for the conditional law ofZZgivenS=sS=s, this target is the equalityPZ∣S=s=PZP\_\{Z\\mid S=s\}=P\_\{Z\}forPSP\_\{S\}\-almost everyss\. We study it whenSSis continuous, as with age, income, or a risk score in domains such as hiring, lending, and criminal justice\(Calders and Verwer,[2010](https://arxiv.org/html/2608.10470#bib.bib3)\), where independence must hold across a continuum of sensitive values rather than across finitely many groups\.

For continuousSS, existing criteria enforce this independence by approximating a localized conditional object indexed byS=sS=s: the conditional lawPZ∣S=sP\_\{Z\\mid S=s\}, or, for generalized demographic parity \(GDP\), the conditional mean𝔼\[⋅∣S=s\]\\mathbb\{E\}\[\\,\\cdot\\mid S=s\]of the prediction head\. They share one form, theSS\-average of a per\-value discrepancyddbetween the conditional law and the marginal,

\(1\)ℐd​\(Z;S\):=𝔼S​\[d​\(PZ∣S,PZ\)\],\\mathcal\{I\}\_\{d\}\(Z;S\):=\\mathbb\{E\}\_\{S\}\\\!\\left\[d\(P\_\{Z\\mid S\},P\_\{Z\}\)\\right\],and differ only indd: GDP uses a first\-moment difference\(Jiang et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib15)\), the expectation\-of\-IPM criterion \(EIPM\) ofKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\)uses an integral probability metric \(IPM\), realized by the kernel\-weighted estimator FREM, and mutual\-information objectives use the Kullback–Leibler divergence\(Cho et al\.,[2020b](https://arxiv.org/html/2608.10470#bib.bib5)\)\. Because a finite sample of sizennhas no repeated observations at an exact valueS=sS=s, each criterion replaces exact conditioning by a localized or density\-based surrogate, such as FREM’s leave\-one\-out kernel\-weighted conditional empirical measure or GDP’s Nadaraya–Watson kernel smoothing onSS\(Nadaraya,[1964](https://arxiv.org/html/2608.10470#bib.bib26); Watson,[1964](https://arxiv.org/html/2608.10470#bib.bib37)\)\. These smoothing\-based estimators carry a localization bandwidth and converge at the one\-dimensional nonparametric rateO​\(n−2/5\)O\(n^\{\-2/5\}\)\.

We propose to enforce independence through a single discrepancy between the joint lawPZ,SP\_\{Z,S\}of the pair\(Z,S\)\(Z,S\)and the product of its marginalsPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}, the product measure on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}; Figure[1](https://arxiv.org/html/2608.10470#S1.F1)contrasts this joint route with the conditional route it replaces\. For a classℱ\\mathcal\{F\}of bounded measurable functions on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}, we measure this discrepancy by the joint\-vs\-product integral probability metric

IPMℱ​\(PZ,S,PZ⊗PS\):=supf∈ℱ\|𝔼PZ,S​\[f\]−𝔼PZ⊗PS​\[f\]\|,\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\bigl\(P\_\{Z,S\},\\,P\_\{Z\}\\otimes P\_\{S\}\\bigr\):=\\sup\_\{f\\in\\mathcal\{F\}\}\\bigl\|\\mathbb\{E\}\_\{P\_\{Z,S\}\}\[f\]\-\\mathbb\{E\}\_\{P\_\{Z\}\\otimes P\_\{S\}\}\[f\]\\bigr\|,which vanishes if and only ifZ⟂SZ\\perp Swheneverℱ\\mathcal\{F\}is rich enough to separate distinct laws \(a characteristic class\), and at this target it agrees with the conditional\-integral criteria of \([1](https://arxiv.org/html/2608.10470#S1.E1)\), namely GDP, EIPM, and mutual information \(Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1)\)\. The averaging over sensitive values is then intrinsic to the joint law rather than an imposed weighting, and no per\-value conditional object enters the definition\.

The link to the conditional\-integral route is not confined to this zero level: the joint discrepancy disintegrates exactly into the same averaged form\. A disintegration identity \(Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)\) rewrites the joint\-vs\-product IPM, for any classℱ\\mathcal\{F\}, as thePSP\_\{S\}\-average of a conditional contrast with the supremum overℱ\\mathcal\{F\}taken outside the average,

IPMℱ​\(PZ,S,PZ⊗PS\)=supf∈ℱ\|∫𝒮∫𝒵f​\(z,s\)​d​\(PZ∣S=s−PZ\)​\(z\)​𝑑PS​\(d​s\)\|\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\bigl\(P\_\{Z,S\},\\,P\_\{Z\}\\otimes P\_\{S\}\\bigr\)=\\sup\_\{f\\in\\mathcal\{F\}\}\\left\|\\int\_\{\\mathcal\{S\}\}\\\!\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,d\\bigl\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\\bigr\)\(z\)\\,dP\_\{S\}\(ds\)\\right\|and on the decomposable classes of Definition[3\.5](https://arxiv.org/html/2608.10470#S3.Thmtheorem5), generated by per\-value witness functions, this average equals the conditional\-integral functionalℐd\\mathcal\{I\}\_\{d\}of \([1](https://arxiv.org/html/2608.10470#S1.E1)\) that GDP and EIPM instantiate \(Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)\)\. Mutual information is the classical precedent for this coincidence: as the Kullback–Leibler discrepancy it equals both the joint\-vs\-product divergenceKL​\(PZ,S∥PZ⊗PS\)\\mathrm\{KL\}\(P\_\{Z,S\}\\,\\\|\\,P\_\{Z\}\\otimes P\_\{S\}\)and thePSP\_\{S\}\-average of the conditional divergencesKL​\(PZ∣S=s∥PZ\)\\mathrm\{KL\}\(P\_\{Z\\mid S=s\}\\,\\\|\\,P\_\{Z\}\)by the chain rule of relative entropy\(Cover and Thomas,[2006](https://arxiv.org/html/2608.10470#bib.bib6), Ch\. 2\), so it already measures independence from the joint law without constructing the conditional family\{PZ∣S=s\}s\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\}\.

Because the target is a single functional of the paired law, it is estimated directly from the joint sample\{\(Zi,Si\)\}i=1n\\\{\(Z\_\{i\},S\_\{i\}\)\\\}\_\{i=1\}^\{n\}using only those paired observations and the witness classℱ\\mathcal\{F\}\. How fast this direct estimate converges then depends onℱ\\mathcal\{F\}rather than on a smoothing bandwidth\.

We take the Hilbert–Schmidt independence criterion \(HSIC\) as the instance obtained by choosingℱ\\mathcal\{F\}to be the unit ball of a product reproducing\-kernel Hilbert space \(RKHS\)\. The empirical HSIC is a closed\-form statistic, computable inO​\(n2\)O\(n^\{2\}\)time from the Gram matrices of\{Zi\}i=1n\\\{Z\_\{i\}\\\}\_\{i=1\}^\{n\}and\{Si\}i=1n\\\{S\_\{i\}\\\}\_\{i=1\}^\{n\}, whose kernel bandwidths are fixed scale parameters rather than per\-value localization for the conditional law\. Being a non\-degenerateVV\-statistic, it converges to its population value at theO​\(n−1/2\)O\(n^\{\-1/2\}\)rate, in contrast to the nonparametricO​\(n−2/5\)O\(n^\{\-2/5\}\)of the smoothing\-based EIPM estimator \(FREM\); our synthetic study fits log–log slopes of−0\.46\-0\.46for HSIC and−0\.44\-0\.44for EIPM, close to−1/2\-1/2and−2/5\-2/5\(§[6\.2](https://arxiv.org/html/2608.10470#S6.SS2), Figure[2](https://arxiv.org/html/2608.10470#S6.F2)\)\. This rate is not only pointwise: up to logarithmic factors it holds uniformly over encoder classes of controlled complexity, such as bounded linear encoders and fixed\-architecture bounded multilayer perceptrons \(Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1), building onNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)\), so the data\-dependent encoder that minimizes the penalty still has small population dependence\. We further prove that HSIC is equivalent to thePSP\_\{S\}\-averaged conditional maximum mean discrepancy \(MMD\) integral up to an explicit spectral tail of the sensitive\-attribute kernel \(Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\)\. As a minibatch regularizer, the resulting objective, which we call FRHSIC, trains about36×36\\timesfaster per epoch than FREM atn=20,000n=20\{,\}000\(§[6\.5](https://arxiv.org/html/2608.10470#S6.SS5)\); penalizing dependence with HSIC for fairness is itself established\(Pérez\-Suay et al\.,[2017](https://arxiv.org/html/2608.10470#bib.bib29); Li et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib20)\), and our contribution is the theory that follows\.

### 1\.1Summary of Contributions

#### A joint\-discrepancy formulation of the continuous\-SScriteria

We show that the existing continuous\-SSfairness criteria arise, on decomposable witness classes, as the conditional reading of a single joint discrepancy\. A disintegration identity \(Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)\) rewrites the joint\-vs\-product IPM as aPSP\_\{S\}\-averaged conditional contrast, and on decomposable witness classes \(Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)\) it coincides with the conditional\-integral functionalℐd\\mathcal\{I\}\_\{d\}of \([1](https://arxiv.org/html/2608.10470#S1.E1)\) that generalized demographic parity and EIPM instantiate, while mutual information is the Kullback–Leibler instance of the sameSS\-averaged form\.

#### A closed\-form HSIC estimator with spectral and uniform control

We give a closed\-formO​\(n2\)O\(n^\{2\}\)estimator of the joint discrepancy, the HSIC obtained from a product RKHS, that is controlled by thePSP\_\{S\}\-averaged conditional object without ever estimating it\. We prove it is equivalent to the conditional MMD integral up to an explicit spectral tail of the sensitive\-attribute kernel \(Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\), bounds the GDP of an RKHS head \(Corollary[4\.10](https://arxiv.org/html/2608.10470#S4.Thmtheorem10)\), and concentrates uniformly at the raten−1/2n^\{\-1/2\}, up to logarithmic factors, under a controlled\-complexity condition on the encoder class \(Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1), building onNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)\)\.

#### A training algorithm for fair representations, FRHSIC

We instantiate the estimator as a minibatch regularizer that attains fairness–accuracy tradeoffs comparable to conditional\-route baselines while training about36×36\\timesfaster per epoch than FREM atn=20,000n=20\{,\}000\(§[6\.5](https://arxiv.org/html/2608.10470#S6.SS5)\)\. On synthetic data and five real datasets, FRHSIC attains comparable fairness–accuracy tradeoffs, shows estimator convergence consistent with the predictedO​\(n−1/2\)O\(n^\{\-1/2\}\)rate \(§[6\.2](https://arxiv.org/html/2608.10470#S6.SS2), Figure[2](https://arxiv.org/html/2608.10470#S6.F2)\), and keeps fairness stable across fresh downstream heads\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x1.png)Figure 1:Conditional and joint routes for continuous\-sensitive fair representation learning\. The conditional route approximates local objects indexed byS=sS=s; the joint route estimates dependence directly from paired samples\.

## 2Preliminaries and Existing Criteria

This section fixes notation, recalls the integral probability metrics used throughout, and shows that the existing continuous\-SSfairness criteria share a common conditional\-integral form, the baseline against which our joint route is compared\.

### 2\.1Setup and representation\-level fairness

Representation\-level fairness asks for a representation that is statistically independent of the sensitive attribute while remaining predictive of the target\. LetX∈𝒳X\\in\\mathcal\{X\}denote the input random vector,S∈𝒮⊆ℝS\\in\\mathcal\{S\}\\subseteq\\mathbb\{R\}the continuous sensitive attribute, andY∈𝒴Y\\in\\mathcal\{Y\}the target variable\. A representation functionh:𝒳→𝒵h:\\mathcal\{X\}\\to\\mathcal\{Z\}yieldsZ=h​\(X\)Z=h\(X\), and a prediction headf:𝒵→𝒴f:\\mathcal\{Z\}\\to\\mathcal\{Y\}yieldsY^=f​\(Z\)=f∘h​\(X\)\\widehat\{Y\}=f\(Z\)=f\\circ h\(X\)\. WritePZP\_\{Z\}for the marginal law ofZZ,PZ∣S=sP\_\{Z\\mid S=s\}for the conditional law ofZZgivenS=sS=s, andPZ,SP\_\{Z,S\}for the joint law\. For a reproducing kernel Hilbert space \(RKHS\)ℱ\\mathcal\{F\}with kernelkk, the mean embedding of a distributionPPisμP=𝔼X∼P​\[k​\(X,⋅\)\]∈ℱ\\mu\_\{P\}=\\mathbb\{E\}\_\{X\\sim P\}\[k\(X,\\cdot\)\]\\in\\mathcal\{F\}\. For binaryS∈\{0,1\}S\\in\\\{0,1\\\}, a predictorY^\\widehat\{Y\}satisfies demographic parity \(DP\) ifY^⟂S\\widehat\{Y\}\\perp S, which for binaryY^\\widehat\{Y\}reduces to𝔼​\[Y^∣S=0\]=𝔼​\[Y^∣S=1\]\\mathbb\{E\}\[\\widehat\{Y\}\\mid S=0\]=\\mathbb\{E\}\[\\widehat\{Y\}\\mid S=1\]\. The goal of fair representation learning is a representationZ=h​\(X\)Z=h\(X\)that is independent ofSSwhile remaining informative aboutYY\. For continuousSSthe exact representation\-level target is

\(2\)Z⟂S,equivalentlyPZ∣S=s=PZfor​PS​\-almost every​sZ\\perp S,\\qquad\\text\{equivalently\}\\qquad P\_\{Z\\mid S=s\}=P\_\{Z\}\\quad\\text\{for \}P\_\{S\}\\text\{\-almost every \}sfor any regular conditional distribution\{PZ∣S=s\}s∈𝒮\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\\in\\mathcal\{S\}\}\. This independence condition is the target that every criterion in this paper enforces; empirical criteria approximate it through a discrepancy betweenPZ∣S=sP\_\{Z\\mid S=s\}andPZP\_\{Z\}, introduced next\.

### 2\.2Discrepancies between probability laws

Distances between distributions in this paper are integral probability metrics, and the maximum mean discrepancy is the kernel instance that, for a characteristic kernel, vanishes exactly when two laws coincide\. For a classℱ\\mathcal\{F\}of bounded measurable functions, the integral probability metric \(IPM\) between distributionsPPandQQisIPMℱ​\(P,Q\):=supf∈ℱ\|𝔼P​\[f\]−𝔼Q​\[f\]\|\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P,Q\):=\\sup\_\{f\\in\\mathcal\{F\}\}\|\\mathbb\{E\}\_\{P\}\[f\]\-\\mathbb\{E\}\_\{Q\}\[f\]\|\. Takingℱ\\mathcal\{F\}to be the unit ball of an RKHS gives the maximum mean discrepancy,

MMD​\(P,Q\)=sup‖f‖ℱ≤1\|𝔼P​\[f\]−𝔼Q​\[f\]\|=‖μP−μQ‖ℱ\.\\mathrm\{MMD\}\(P,Q\)=\\sup\_\{\\\|f\\\|\_\{\\mathcal\{F\}\}\\leq 1\}\|\\mathbb\{E\}\_\{P\}\[f\]\-\\mathbb\{E\}\_\{Q\}\[f\]\|=\\\|\\mu\_\{P\}\-\\mu\_\{Q\}\\\|\_\{\\mathcal\{F\}\}\.Whenkkis characteristic,MMD​\(P,Q\)=0\\mathrm\{MMD\}\(P,Q\)=0if and only ifP=QP=Q\(Gretton et al\.,[2012](https://arxiv.org/html/2608.10470#bib.bib14)\)\. Section[3](https://arxiv.org/html/2608.10470#S3)applies the IPM withP=PZ,SP=P\_\{Z,S\}andQ=PZ⊗PSQ=P\_\{Z\}\\otimes P\_\{S\}\.

### 2\.3Conditional\-integral criteria for continuous sensitive attributes

Existing continuous\-SSfairness criteria share one form, theSS\-average of a per\-value discrepancy between the conditional and marginal laws ofZZ, and differ only in that discrepancy\. For a discrepancyddbetween distributions, the integral functional is

ℐd​\(Z;S\):=𝔼S​\[d​\(PZ∣S,PZ\)\],\\mathcal\{I\}\_\{d\}\(Z;S\):=\\mathbb\{E\}\_\{S\}\\\!\\left\[d\(P\_\{Z\\mid S\},P\_\{Z\}\)\\right\],which is well\-defined for Polish𝒵,𝒮\\mathcal\{Z\},\\mathcal\{S\}once a regular conditional probability is selected; existence follows from the disintegration theorem\(Kallenberg,[2002](https://arxiv.org/html/2608.10470#bib.bib16), Theorem 5\.4\)\. Three instances recur\. Generalized demographic parity \(GDP\)\(Jiang et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib15)\)takesddto be a first\-moment difference evaluated through a measurable headf:𝒵→ℝf:\\mathcal\{Z\}\\to\\mathbb\{R\},

\(3\)ΔGDP\(f\)=𝔼S\|𝔼Z\[f\(Z\)∣S\]−𝔼Z\[f\(Z\)\]\|,Z=h\(X\),\\Delta\_\{\\mathrm\{GDP\}\}\(f\)=\\mathbb\{E\}\_\{S\}\\left\|\\mathbb\{E\}\_\{Z\}\[f\(Z\)\\mid S\]\-\\mathbb\{E\}\_\{Z\}\[f\(Z\)\]\\right\|,\\qquad Z=h\(X\),with the conditional expectation estimated by Nadaraya–Watson kernel smoothing onSS\(Nadaraya,[1964](https://arxiv.org/html/2608.10470#bib.bib26); Watson,[1964](https://arxiv.org/html/2608.10470#bib.bib37)\); GDP reduces to DP whenSSis binary\. The expectation of IPM \(EIPM\)\(Kong et al\.,[2025](https://arxiv.org/html/2608.10470#bib.bib18)\)takesddto be an IPM,

EIPM𝒱​\(Z;S\):=𝔼S​\[IPM𝒱​\(PZ∣S,PZ\)\],IPM𝒱​\(P,Q\)=supf∈𝒱\|𝔼P​\[f\]−𝔼Q​\[f\]\|,\\mathrm\{EIPM\}\_\{\\mathcal\{V\}\}\(Z;S\):=\\mathbb\{E\}\_\{S\}\[\\mathrm\{IPM\}\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\],\\qquad\\mathrm\{IPM\}\_\{\\mathcal\{V\}\}\(P,Q\)=\\sup\_\{f\\in\\mathcal\{V\}\}\|\\mathbb\{E\}\_\{P\}\[f\]\-\\mathbb\{E\}\_\{Q\}\[f\]\|,and controlling EIPM controls GDP\(Kong et al\.,[2025](https://arxiv.org/html/2608.10470#bib.bib18)\)\. Mutual information is the Kullback–Leibler instance,

I​\(Z;S\)=𝔼S​\[KL​\(PZ∣S∥PZ\)\],I\(Z;S\)=\\mathbb\{E\}\_\{S\}\\\!\\left\[\\mathrm\{KL\}\(P\_\{Z\\mid S\}\\,\\\|\\,P\_\{Z\}\)\\right\],and characterizes independence,I​\(Z;S\)=0⇔Z⟂SI\(Z;S\)=0\\iff Z\\perp S\(Cover and Thomas,[2006](https://arxiv.org/html/2608.10470#bib.bib6), Theorem 2\.6\.3\)\. Section[3](https://arxiv.org/html/2608.10470#S3)relates this conditional\-integral route to a single joint\-vs\-product discrepancy through disintegration, recovering the conditional\-integral IPM on decomposable witness classes\.

### 2\.4Joint dependence measures and HSIC

An alternative measures dependence directly between the joint law and the product of marginals, and the Hilbert–Schmidt independence criterion is the kernel instance that, under characteristic product kernels, vanishes exactly under independence\. This is the object the method of this paper is built on\.

###### Definition 2\.1\(Mean embeddings and HSIC\)\.

Letℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}andℱ𝒮\\mathcal\{F\}\_\{\\mathcal\{S\}\}be RKHSs on𝒵\\mathcal\{Z\}and𝒮\\mathcal\{S\}with bounded characteristic kernelsk𝒵k\_\{\\mathcal\{Z\}\}andk𝒮k\_\{\\mathcal\{S\}\}, and letℱ𝒵⊗𝒮\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}be the tensor\-product RKHS on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}with product kernelk𝒵⊗k𝒮k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}\. The marginal and joint mean embeddings are

μZ:=𝔼Z​\[k𝒵​\(Z,⋅\)\]∈ℱ𝒵,μS:=𝔼S​\[k𝒮​\(S,⋅\)\]∈ℱ𝒮,\\mu\_\{Z\}:=\\mathbb\{E\}\_\{Z\}\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\]\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\},\\qquad\\mu\_\{S\}:=\\mathbb\{E\}\_\{S\}\[k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\},μZ,S:=𝔼\(Z,S\)​\[k𝒵​\(Z,⋅\)⊗k𝒮​\(S,⋅\)\]∈ℱ𝒵⊗𝒮,\\mu\_\{Z,S\}:=\\mathbb\{E\}\_\{\(Z,S\)\}\\bigl\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\bigr\]\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\},andμZ⊗μS∈ℱ𝒵⊗𝒮\\mu\_\{Z\}\\otimes\\mu\_\{S\}\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}is the embedding of the product lawPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}\. The conditional mean embedding ofZZgivenS=sS=sisμZ∣S=s:=𝔼​\[k𝒵​\(Z,⋅\)∣S=s\]∈ℱ𝒵\\mu\_\{Z\\mid S=s\}:=\\mathbb\{E\}\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\\mid S=s\]\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}; we writeμZ∣S\\mu\_\{Z\\mid S\}for theℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\-valued random elements↦μZ∣S=ss\\mapsto\\mu\_\{Z\\mid S=s\}evaluated atSS\. The Hilbert–Schmidt independence criterion \(HSIC\) is the squared maximum mean discrepancy between the joint law and the product of marginals,

HSIC​\(Z,S\):=‖μZ,S−μZ⊗μS‖ℱ𝒵⊗𝒮2\.\\mathrm\{HSIC\}\(Z,S\):=\\bigl\\\|\\mu\_\{Z,S\}\-\\mu\_\{Z\}\\otimes\\mu\_\{S\}\\bigr\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}\.

Under characteristic product kernels,HSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0if and only ifZ⟂SZ\\perp S\(Gretton et al\.,[2005](https://arxiv.org/html/2608.10470#bib.bib12)\)\. Section[3](https://arxiv.org/html/2608.10470#S3)relates this joint discrepancy to conditional\-integral criteria through a disintegration identity\. The equality with the conditional\-integral IPM requires the decomposability condition stated in Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)\.

## 3The Joint Discrepancy: An Intrinsic Metric for the Conditional\-Integral Criteria

### 3\.1Zero\-level relation to existing criteria

This section develops the central object of this paper: a single joint discrepancy betweenPZ,SP\_\{Z,S\}andPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}that serves as an intrinsic metric for the conditional\-integral criteria\. We begin at the zero level, where a characteristic witness class makes the joint discrepancy vanish exactly at the representation\-level fairness target shared by the existing criteria; the nonzero regime is developed in §[3\.2](https://arxiv.org/html/2608.10470#S3.SS2)–§[4\.2](https://arxiv.org/html/2608.10470#S4.SS2)\.

###### Proposition 3\.1\(Zero\-level characterization of representation fairness\)\.

Let𝒵\\mathcal\{Z\}and𝒮\\mathcal\{S\}be Polish spaces, letℱ\\mathcal\{F\}be a characteristic class of bounded measurable functions on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}, that is, one for whichIPMℱ\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}separates probability laws on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}, and let\(Z,S\)\(Z,S\)be a random pair on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}\. Then the following are equivalent:

1. 1\.IPMℱ​\(PZ,S,PZ⊗PS\)=0\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P\_\{Z,S\},\\,P\_\{Z\}\\otimes P\_\{S\}\)=0;
2. 2\.Z⟂SZ\\perp S, equivalentlyPZ,S=PZ⊗PSP\_\{Z,S\}=P\_\{Z\}\\otimes P\_\{S\};
3. 3\.for some, equivalently every, regular conditional distribution\{PZ∣S=s\}s∈𝒮\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\\in\\mathcal\{S\}\}, PZ∣S=s=PZfor​PS​\-a\.e\.​s;P\_\{Z\\mid S=s\}=P\_\{Z\}\\quad\\text\{for \}P\_\{S\}\\text\{\-a\.e\. \}s;
4. 4\.for every bounded measurableg:𝒵→ℝg:\\mathcal\{Z\}\\to\\mathbb\{R\}, 𝔼​\[g​\(Z\)∣S=s\]=𝔼​\[g​\(Z\)\]for​PS​\-a\.e\.​s\.\\mathbb\{E\}\[g\(Z\)\\mid S=s\]=\\mathbb\{E\}\[g\(Z\)\]\\quad\\text\{for \}P\_\{S\}\\text\{\-a\.e\. \}s\.

###### Proof 3\.2\.

The equivalences are proved in Appendix[A\.1](https://arxiv.org/html/2608.10470#A1.SS1): condition \(1\) is equivalent to \(2\) because a characteristic class separatesPZ,SP\_\{Z,S\}fromPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}, and \(2\)–\(4\) are equivalent by disintegration\. The MMD\-based EIPM instance,EIPM𝒱​\(Z;S\)=0\\mathrm\{EIPM\}\_\{\\mathcal\{V\}\}\(Z;S\)=0for the unit ball𝒱⊂ℱ𝒵\\mathcal\{V\}\\subset\\mathcal\{F\}\_\{\\mathcal\{Z\}\}, and the downstream\-head \(GDP\) form are recorded in Appendices[A\.2](https://arxiv.org/html/2608.10470#A1.SS2)–[A\.3](https://arxiv.org/html/2608.10470#A1.SS3)\.

Remark \(zero\-level equivalence only\)\.Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1)is a zero\-level statement\. Away from zero the joint discrepancy and the conditional\-integral criteria are different functionals: the conditional\-integral criteria average a per\-value discrepancy betweenPZ∣S=sP\_\{Z\\mid S=s\}andPZP\_\{Z\}over the law ofSS, whereas the joint discrepancy applies a single supremum to the paired law\.

### 3\.2Conditional\-integral and joint formulations

Conditional\-integral criteria comparePZ∣S=sP\_\{Z\\mid S=s\}withPZP\_\{Z\}and average the comparison overSS, whereas the joint route compares the joint lawPZ,SP\_\{Z,S\}with the product of marginalsPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}\. WhenSStakes finitely many values the two formulations coincide as aggregated two\-sample MMDs, recorded as Lemma[B\.3](https://arxiv.org/html/2608.10470#A2.Thmtheorem3)in Appendix[B\.1](https://arxiv.org/html/2608.10470#A2.SS1)\. The case of practical interest is continuousSS, where the conditional family\{PZ∣S=s\}s\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\}is uncountable and the two formulations differ in how they are estimated\. The next theorem disintegrates the joint\-vs\-product IPM into aPSP\_\{S\}\-weighted conditional contrast\.

###### Theorem 3\.3\(Disintegration of the joint\-vs\-product IPM\)\.

Let𝒵\\mathcal\{Z\}and𝒮\\mathcal\{S\}be standard Borel spaces, and let\{PZ∣S=s\}s∈𝒮\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\\in\\mathcal\{S\}\}be a regular conditional distribution ofZZgivenSS\. For any classℱ\\mathcal\{F\}of bounded measurable functions on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\},

IPMℱ​\(PZ,S,PZ⊗PS\)=supf∈ℱ\|∫𝒮∫𝒵f​\(z,s\)​d​\(PZ∣S=s−PZ\)​\(z\)​𝑑PS​\(d​s\)\|\.\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\bigl\(P\_\{Z,S\},\\,P\_\{Z\}\\otimes P\_\{S\}\\bigr\)=\\sup\_\{f\\in\\mathcal\{F\}\}\\left\|\\int\_\{\\mathcal\{S\}\}\\\!\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,d\\bigl\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\\bigr\)\(z\)\\,dP\_\{S\}\(ds\)\\right\|\.

###### Proof 3\.4\.

See Appendix[A\.4](https://arxiv.org/html/2608.10470#A1.SS4)\.

The theorem shows that thePSP\_\{S\}weighting arises from disintegrating the joint law, rather than from an external weighting choice\. For any classℱ\\mathcal\{F\}, it places a single supremum outside thePSP\_\{S\}\-average of the conditional contrast; this average equals the conditional\-integral functionalℐd\\mathcal\{I\}\_\{d\}exactly whenℱ\\mathcal\{F\}decomposes acrossss, which Definition[3\.5](https://arxiv.org/html/2608.10470#S3.Thmtheorem5)and Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)make precise\.

###### Definition 3\.5\(Decomposable witness class\)\.

Let𝒱\\mathcal\{V\}be a symmetric class of bounded measurable functions on𝒵\\mathcal\{Z\}, meaningg∈𝒱g\\in\\mathcal\{V\}implies−g∈𝒱\-g\\in\\mathcal\{V\}\. A classℱ\\mathcal\{F\}of bounded measurable functions on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}is decomposable over𝒱\\mathcal\{V\}if:

1. 1\.for every measurable selectors↦gs∈𝒱s\\mapsto g\_\{s\}\\in\\mathcal\{V\}, the function\(z,s\)↦gs​\(z\)\(z,s\)\\mapsto g\_\{s\}\(z\)belongs toℱ\\mathcal\{F\};
2. 2\.for everyf∈ℱf\\in\\mathcal\{F\}, the slicef​\(⋅,s\)f\(\\cdot,s\)belongs to𝒱\\mathcal\{V\}forPSP\_\{S\}\-almost everyss\.

###### Corollary 3\.6\(Recovery of the conditional\-integral IPM under decomposability\)\.

Adopt the assumptions of Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3):𝒵\\mathcal\{Z\}and𝒮\\mathcal\{S\}are standard Borel spaces and\{PZ∣S=s\}s∈𝒮\\\{P\_\{Z\\mid S=s\}\\\}\_\{s\\in\\mathcal\{S\}\}is a regular conditional distribution ofZZgivenSS\. Let𝒱\\mathcal\{V\}be a symmetric uniformly bounded class of measurable functions on𝒵\\mathcal\{Z\}, and define

d𝒱​\(P,Q\):=IPM𝒱​\(P,Q\)\.d\_\{\\mathcal\{V\}\}\(P,Q\):=\\mathrm\{IPM\}\_\{\\mathcal\{V\}\}\(P,Q\)\.Letℱ\\mathcal\{F\}be decomposable over𝒱\\mathcal\{V\}in the sense of Definition[3\.5](https://arxiv.org/html/2608.10470#S3.Thmtheorem5)\. Assume that for everyε\>0\\varepsilon\>0there exists a measurable selectors↦gsε∈𝒱s\\mapsto g\_\{s\}^\{\\varepsilon\}\\in\\mathcal\{V\}such that

∫gsε​\(z\)​d​\(PZ∣S=s−PZ\)​\(z\)≥d𝒱​\(PZ∣S=s,PZ\)−εfor​PS​\-a\.e\.​s\.\\int g\_\{s\}^\{\\varepsilon\}\(z\)\\,d\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\)\(z\)\\geq d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\-\\varepsilon\\quad\\text\{for \}P\_\{S\}\\text\{\-a\.e\. \}s\.Then

IPMℱ​\(PZ,S,PZ⊗PS\)=𝔼S​\[d𝒱​\(PZ∣S=s,PZ\)\]=ℐd𝒱​\(Z;S\)\.\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\)=\\mathbb\{E\}\_\{S\}\\\!\\left\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\\right\]=\\mathcal\{I\}\_\{d\_\{\\mathcal\{V\}\}\}\(Z;S\)\.

###### Proof 3\.7\.

See Appendix[A\.5](https://arxiv.org/html/2608.10470#A1.SS5)\.

Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)therefore recovers the conditional\-integral IPM, which GDP and EIPM instantiate, exactly when the witness class decomposes acrossss; mutual information is the Kullback–Leibler instance of the sameSS\-averaged form\.

## 4HSIC: A Closed\-Form Equivalent with Demographic\-Parity Control

HSIC is equivalent, up to an explicit spectral tail, to the conditional MMD integral it replaces, and that equivalence carries over to the fairness metric itself\. §[4\.1](https://arxiv.org/html/2608.10470#S4.SS1)proves the equivalence with the conditional MMD integral, the conditional\-integral criterion HSIC replaces\. §[4\.2](https://arxiv.org/html/2608.10470#S4.SS2)turns it into control of the demographic\-parity gap at the population level and on a finite sample\.

### 4\.1Equivalence with the conditional MMD integral

HSIC is the kernel joint discrepancy obtained by comparingPZ,SP\_\{Z,S\}withPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}\. Instantiating the joint IPM of Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)with the unit ball of the tensor\-product RKHSℱ𝒵⊗𝒮\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}of Definition[2\.1](https://arxiv.org/html/2608.10470#S2.Thmtheorem1)gives the maximum mean discrepancyMMDk𝒵⊗k𝒮​\(PZ,S,PZ⊗PS\)\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}\}\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\), and HSIC is its square,

HSIC​\(Z,S\)=MMDk𝒵⊗k𝒮2​\(PZ,S,PZ⊗PS\)\.\\mathrm\{HSIC\}\(Z,S\)=\\mathrm\{MMD\}^\{2\}\_\{k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}\}\\bigl\(P\_\{Z,S\},\\,P\_\{Z\}\\otimes P\_\{S\}\\bigr\)\.We assume throughout that the product kernelk𝒵⊗k𝒮k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}is characteristic to the relevant joint and product measures on𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}; this holds under standard conditions for the Gaussian kernels used in our experiments\. The first of the two bounds comes from a decomposition\. The following proposition writes HSIC over the conditional deviationsμZ∣S=s−μZ\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}and bounds it by the squared expected MMD, so a small conditional MMD makes HSIC small\.

###### Proposition 4\.1\(Decomposition of HSIC and its expected\-MMD bound\)\.

Supposek𝒵k\_\{\\mathcal\{Z\}\}andk𝒮k\_\{\\mathcal\{S\}\}are bounded, i\.e\.,k𝒵​\(z,z\)≤κ𝒵k\_\{\\mathcal\{Z\}\}\(z,z\)\\leq\\kappa\_\{\\mathcal\{Z\}\}andk𝒮​\(s,s\)≤κ𝒮k\_\{\\mathcal\{S\}\}\(s,s\)\\leq\\kappa\_\{\\mathcal\{S\}\}for allz∈𝒵z\\in\\mathcal\{Z\},s∈𝒮s\\in\\mathcal\{S\}\. Then HSIC admits the decomposition

\(4\)HSIC​\(Z,S\)=𝔼S,S′​\[⟨μZ\|S−μZ,μZ\|S′−μZ⟩ℱ𝒵⋅k𝒮​\(S,S′\)\],\\mathrm\{HSIC\}\(Z,S\)=\\mathbb\{E\}\_\{S,S^\{\\prime\}\}\\left\[\\left\\langle\\mu\_\{Z\|S\}\-\\mu\_\{Z\},\\mu\_\{Z\|S^\{\\prime\}\}\-\\mu\_\{Z\}\\right\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\cdot k\_\{\\mathcal\{S\}\}\(S,S^\{\\prime\}\)\\right\],whereS,S′S,S^\{\\prime\}are independent copies\. Moreover, withMMDk𝒵​\(PZ∣S=s,PZ\)=‖μZ∣S=s−μZ‖ℱ𝒵\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)=\\\|\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}the maximum mean discrepancy of §[2](https://arxiv.org/html/2608.10470#S2)underk𝒵k\_\{\\mathcal\{Z\}\}, HSIC is bounded by the squared expected MMD:

\(5\)HSIC​\(Z,S\)≤κ𝒮⋅\(𝔼S​\[MMDk𝒵​\(PZ∣S,PZ\)\]\)2\.\\mathrm\{HSIC\}\(Z,S\)\\leq\\kappa\_\{\\mathcal\{S\}\}\\cdot\\left\(\\mathbb\{E\}\_\{S\}\\\!\\left\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\\bigl\(P\_\{Z\\mid S\},P\_\{Z\}\\bigr\)\\right\]\\right\)^\{2\}\.

###### Proof 4\.2\.

See Appendix[A\.6](https://arxiv.org/html/2608.10470#A1.SS6)\.

The second bound comes from the same decomposition\. Because‖μZ∣S=s−μZ‖ℱ𝒵=MMDk𝒵​\(PZ∣S=s,PZ\)\\\|\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}=\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\), the conditional deviationΔ​\(s\):=μZ∣S=s−μZ\\Delta\(s\):=\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}is the conditional MMD in vector form\. The next theorem shows that a small HSIC makes every RKHS\-smoothed average ofΔ\\Deltasmall, so the joint statistic controls the conditional MMD after smoothing\.

###### Theorem 4\.4\(Smoothed conditional\-distribution control by HSIC\)\.

Letk𝒵k\_\{\\mathcal\{Z\}\}andk𝒮k\_\{\\mathcal\{S\}\}be bounded measurable kernels with RKHSsℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}andℱ𝒮\\mathcal\{F\}\_\{\\mathcal\{S\}\}\. Let

Δ​\(s\):=μZ∣S=s−μZ∈ℱ𝒵,μZ∣S=s:=𝔼​\[k𝒵​\(Z,⋅\)∣S=s\]\.\\Delta\(s\):=\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\},\\qquad\\mu\_\{Z\\mid S=s\}:=\\mathbb\{E\}\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\\mid S=s\]\.ThenΔ∈L2​\(PS;ℱ𝒵\)\\Delta\\in L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)and

HSIC​\(Z,S\)=‖𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]‖ℱ𝒵⊗𝒮2\.\\mathrm\{HSIC\}\(Z,S\)=\\bigl\\\|\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}^\{2\}\.Moreover, for everyg∈ℱ𝒮g\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\},

\(6\)‖𝔼S​\[g​\(S\)​Δ​\(S\)\]‖ℱ𝒵2≤‖g‖ℱ𝒮2​HSIC​\(Z,S\)\.\\bigl\\\|\\mathbb\{E\}\_\{S\}\[g\(S\)\\Delta\(S\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\\|g\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\\,\\mathrm\{HSIC\}\(Z,S\)\.

###### Proof 4\.5\.

See Appendix[A\.7](https://arxiv.org/html/2608.10470#A1.SS7)\.

At a single sensitive values0s\_\{0\}, the same bound controls the conditional MMD localized nears0s\_\{0\}, the contrast a per\-value estimator would target\.

###### Corollary 4\.6\(Localized conditional\-MMD control by HSIC\)\.

Under the assumptions of Theorem[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4), for anys0∈𝒮s\_\{0\}\\in\\mathcal\{S\}withk𝒮​\(s0,s0\)\>0k\_\{\\mathcal\{S\}\}\(s\_\{0\},s\_\{0\}\)\>0,

‖𝔼S​\[k𝒮​\(S,s0\)​\(μZ∣S−μZ\)\]‖ℱ𝒵2≤k𝒮​\(s0,s0\)​HSIC​\(Z,S\)\.\\bigl\\\|\\mathbb\{E\}\_\{S\}\[k\_\{\\mathcal\{S\}\}\(S,s\_\{0\}\)\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq k\_\{\\mathcal\{S\}\}\(s\_\{0\},s\_\{0\}\)\\,\\mathrm\{HSIC\}\(Z,S\)\.More generally, for anys0,s1∈𝒮s\_\{0\},s\_\{1\}\\in\\mathcal\{S\},

‖𝔼S​\[\{k𝒮​\(S,s0\)−k𝒮​\(S,s1\)\}​\(μZ∣S−μZ\)\]‖ℱ𝒵2≤‖k𝒮​\(s0,⋅\)−k𝒮​\(s1,⋅\)‖ℱ𝒮2​HSIC​\(Z,S\)\.\\displaystyle\\bigl\\\|\\mathbb\{E\}\_\{S\}\[\\\{k\_\{\\mathcal\{S\}\}\(S,s\_\{0\}\)\-k\_\{\\mathcal\{S\}\}\(S,s\_\{1\}\)\\\}\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\\|k\_\{\\mathcal\{S\}\}\(s\_\{0\},\\cdot\)\-k\_\{\\mathcal\{S\}\}\(s\_\{1\},\\cdot\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\\mathrm\{HSIC\}\(Z,S\)\.

###### Proof 4\.7\.

See Appendix[A\.8](https://arxiv.org/html/2608.10470#A1.SS8)\.

ResolvingΔ\\Deltaalong the spectrum of the sensitive\-attribute kernel turns the control of Theorem[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4)into a bound on the conditional MMD integral; combined with the bound of Proposition[4\.1](https://arxiv.org/html/2608.10470#S4.Thmtheorem1), the two give a two\-sided equivalence: HSIC and the conditional MMD integral control each other up to an explicit spectral tail, so making the closed\-form statistic small makes the conditional MMD integral that EIPM targets small up to the residual tailρm2\\rho\_\{m\}^\{2\}\.

###### Theorem 4\.8\(Spectral equivalence of HSIC and the conditional MMD integral\)\.

Assume the conditions of Theorem[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4), and letκ𝒮\\kappa\_\{\\mathcal\{S\}\}be the bound onk𝒮k\_\{\\mathcal\{S\}\}from Proposition[4\.1](https://arxiv.org/html/2608.10470#S4.Thmtheorem1)\. LetΔ​\(s\):=μZ∣S=s−μZ\\Delta\(s\):=\\mu\_\{Z\\mid S=s\}\-\\mu\_\{Z\}and letTS:L2​\(PS\)→L2​\(PS\)T\_\{S\}:L^\{2\}\(P\_\{S\}\)\\to L^\{2\}\(P\_\{S\}\)be the kernel integral operator

\(TS​u\)​\(s\):=∫k𝒮​\(s,t\)​u​\(t\)​𝑑PS​\(t\),\(T\_\{S\}u\)\(s\):=\\int k\_\{\\mathcal\{S\}\}\(s,t\)u\(t\)\\,dP\_\{S\}\(t\),with orthonormal eigensystem\(λℓ,ψℓ\)ℓ≥1\(\\lambda\_\{\\ell\},\\psi\_\{\\ell\}\)\_\{\\ell\\geq 1\},λ1≥λ2≥⋯≥0\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\cdots\\geq 0, and letPmP\_\{m\}denote theL2​\(PS\)L^\{2\}\(P\_\{S\}\)projection ontospan​\{ψ1,…,ψm\}\\mathrm\{span\}\\\{\\psi\_\{1\},\\ldots,\\psi\_\{m\}\\\}, extended componentwise toL2​\(PS;ℱ𝒵\)L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\. Then, for anymmwithλm\>0\\lambda\_\{m\}\>0such that‖\(I−Pm\)​Δ‖L2​\(PS;ℱ𝒵\)2≤ρm2\\\|\(I\-P\_\{m\}\)\\Delta\\\|\_\{L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\}^\{2\}\\leq\\rho\_\{m\}^\{2\},

1κ𝒮​HSIC​\(Z,S\)≤𝔼S​\[MMDk𝒵2​\(PZ∣S,PZ\)\]≤1λm​HSIC​\(Z,S\)\+ρm2\.\\frac\{1\}\{\\kappa\_\{\\mathcal\{S\}\}\}\\,\\mathrm\{HSIC\}\(Z,S\)\\;\\leq\\;\\mathbb\{E\}\_\{S\}\\\!\\left\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}^\{2\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\\right\]\\;\\leq\\;\\frac\{1\}\{\\lambda\_\{m\}\}\\,\\mathrm\{HSIC\}\(Z,S\)\+\\rho\_\{m\}^\{2\}\.HenceHSIC​\(Z,S\)\\mathrm\{HSIC\}\(Z,S\)and the conditional MMD integral control each other up to the constantsκ𝒮,λm\\kappa\_\{\\mathcal\{S\}\},\\lambda\_\{m\}and the explicit spectral tailρm2\\rho\_\{m\}^\{2\}\.

###### Proof 4\.9\.

The upper bound is the spectral argument of Appendix[A\.9](https://arxiv.org/html/2608.10470#A1.SS9): with𝔼S​\[MMDk𝒵2​\(PZ∣S,PZ\)\]=‖Δ‖L2​\(PS;ℱ𝒵\)2\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}^\{2\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]=\\\|\\Delta\\\|\_\{L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\}^\{2\}, expandingΔ\\Deltain the eigenbasis ofTST\_\{S\}gives‖Δ‖L2​\(PS;ℱ𝒵\)2≤λm−1​HSIC​\(Z,S\)\+ρm2\\\|\\Delta\\\|\_\{L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\}^\{2\}\\leq\\lambda\_\{m\}^\{\-1\}\\mathrm\{HSIC\}\(Z,S\)\+\\rho\_\{m\}^\{2\}\. For the lower bound, Proposition[4\.1](https://arxiv.org/html/2608.10470#S4.Thmtheorem1)givesHSIC​\(Z,S\)≤κ𝒮​\(𝔼S​\[MMDk𝒵​\(PZ∣S,PZ\)\]\)2\\mathrm\{HSIC\}\(Z,S\)\\leq\\kappa\_\{\\mathcal\{S\}\}\\bigl\(\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]\\bigr\)^\{2\}, and Jensen’s inequality gives\(𝔼S​\[MMDk𝒵​\(PZ∣S,PZ\)\]\)2≤𝔼S​\[MMDk𝒵2​\(PZ∣S,PZ\)\]\\bigl\(\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]\\bigr\)^\{2\}\\leq\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}^\{2\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]\.

HSIC controls the conditional\-distribution deviations lying in the well\-conditioned spectral directions ofTST\_\{S\}\. Because the kernel integral operator on𝒮\\mathcal\{S\}is compact, the projected tailρm\\rho\_\{m\}decreases inmmand the remaining directions enter only through it, so whenρm\\rho\_\{m\}is small, small HSIC implies small conditional\-integral MMD\. This is the population counterpart of the finite\-sample spectral factorλ^m−1\\hat\{\\lambda\}\_\{m\}^\{\-1\}in Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\.

### 4\.2Control of the demographic\-parity gap

The equivalence transfers to the fairness metric, in population and then on a finite sample\. At the population level, the same spectral bound reaches generalized demographic parity: for an RKHS prediction head, small HSIC controlsΔGDP​\(f\)\\Delta\_\{\\mathrm\{GDP\}\}\(f\)up to the spectral tail of its conditional meanmfm\_\{f\}\.

###### Corollary 4\.10\(Spectral control of GDP for RKHS heads\)\.

Under the assumptions of Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8), letf∈ℱ𝒵f\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}and define

mf​\(s\):=𝔼​\[f​\(Z\)∣S=s\]−𝔼​\[f​\(Z\)\]\.m\_\{f\}\(s\):=\\mathbb\{E\}\[f\(Z\)\\mid S=s\]\-\\mathbb\{E\}\[f\(Z\)\]\.LetPmP\_\{m\}be the spectral projection from Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\. Then

‖Pm​mf‖L2​\(PS\)2≤‖f‖ℱ𝒵2λm​HSIC​\(Z,S\)\.\\\|P\_\{m\}m\_\{f\}\\\|\_\{L^\{2\}\(P\_\{S\}\)\}^\{2\}\\leq\\frac\{\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\}\{\\lambda\_\{m\}\}\\,\\mathrm\{HSIC\}\(Z,S\)\.If‖\(I−Pm\)​mf‖L2​\(PS\)2≤rm,f2\\\|\(I\-P\_\{m\}\)m\_\{f\}\\\|\_\{L^\{2\}\(P\_\{S\}\)\}^\{2\}\\leq r\_\{m,f\}^\{2\}, then

ΔGDP​\(f\)2=\(𝔼S​\|mf​\(S\)\|\)2≤‖f‖ℱ𝒵2λm​HSIC​\(Z,S\)\+rm,f2\.\\Delta\_\{\\mathrm\{GDP\}\}\(f\)^\{2\}=\\bigl\(\\mathbb\{E\}\_\{S\}\|m\_\{f\}\(S\)\|\\bigr\)^\{2\}\\leq\\frac\{\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\}\{\\lambda\_\{m\}\}\\,\\mathrm\{HSIC\}\(Z,S\)\+r\_\{m,f\}^\{2\}\.

###### Proof 4\.11\.

See Appendix[A\.10](https://arxiv.org/html/2608.10470#A1.SS10)\.

On a finite sample, the same control holds for the gap computed from data\. The next theorem is the finite\-sample form of Corollary[4\.10](https://arxiv.org/html/2608.10470#S4.Thmtheorem10): the empirical HSIC statistic controls the empirical demographic\-parity gap of any RKHS head, with the eigenvalueλ^m\\hat\{\\lambda\}\_\{m\}of the centered sensitive Gram matrixL~\\widetilde\{L\}in the role ofλm\\lambda\_\{m\}\.

###### Theorem 4\.12\(Finite\-sample control of the empirical demographic\-parity gap by HSIC\)\.

Let\{\(Zi,Si\)\}i=1n\\\{\(Z\_\{i\},S\_\{i\}\)\\\}\_\{i=1\}^\{n\}be a sample\. LetK,L∈ℝn×nK,L\\in\\mathbb\{R\}^\{n\\times n\}be the Gram matrices

Ki​j=k𝒵​\(Zi,Zj\),Li​j=k𝒮​\(Si,Sj\),K\_\{ij\}=k\_\{\\mathcal\{Z\}\}\(Z\_\{i\},Z\_\{j\}\),\\qquad L\_\{ij\}=k\_\{\\mathcal\{S\}\}\(S\_\{i\},S\_\{j\}\),and let

H:=In−n−1​𝟏𝟏⊤,K~:=H​K​H,L~:=H​L​H\.H:=I\_\{n\}\-n^\{\-1\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\},\\qquad\\widetilde\{K\}:=HKH,\\qquad\\widetilde\{L\}:=HLH\.Define the biased empirical HSIC statistic

\(7\)HSIC^n​\(Z,S\):=n−2​tr⁡\(K~​L~\)\.\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(Z,S\):=n^\{\-2\}\\operatorname\{tr\}\(\\widetilde\{K\}\\widetilde\{L\}\)\.Letλ^1≥⋯≥λ^r\>0\\hat\{\\lambda\}\_\{1\}\\geq\\cdots\\geq\\hat\{\\lambda\}\_\{r\}\>0be the positive eigenvalues ofn−1​L~n^\{\-1\}\\widetilde\{L\}with orthonormal eigenvectorse^1,…,e^r∈ℝn\\hat\{e\}\_\{1\},\\ldots,\\hat\{e\}\_\{r\}\\in\\mathbb\{R\}^\{n\}, wherer=rank⁡\(L~\)r=\\operatorname\{rank\}\(\\widetilde\{L\}\), and for1≤m≤r1\\leq m\\leq rletPmP\_\{m\}be the orthogonal projection ontospan⁡\{e^1,…,e^m\}\\operatorname\{span\}\\\{\\hat\{e\}\_\{1\},\\ldots,\\hat\{e\}\_\{m\}\\\}\. Forf∈ℱ𝒵f\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}, define

f¯n:=n−1​∑j=1nf​\(Zj\),δ^f,i:=f​\(Zi\)−f¯n,𝜹f:=\(δ^f,1,…,δ^f,n\)⊤,\\bar\{f\}\_\{n\}:=n^\{\-1\}\\sum\_\{j=1\}^\{n\}f\(Z\_\{j\}\),\\qquad\\hat\{\\delta\}\_\{f,i\}:=f\(Z\_\{i\}\)\-\\bar\{f\}\_\{n\},\\qquad\\bm\{\\delta\}\_\{f\}:=\(\\hat\{\\delta\}\_\{f,1\},\\ldots,\\hat\{\\delta\}\_\{f,n\}\)^\{\\top\},and letm^f:=Pm​𝛅f\\widehat\{m\}\_\{f\}:=P\_\{m\}\\bm\{\\delta\}\_\{f\}be the empirical conditional\-mean gap offf, the prediction’s dependence onSSresolved on themmbest\-conditioned sensitive directions\. Then the empirical demographic\-parity gapΔ^GDP​\(f\):=n−1​∑i=1n\|\(m^f\)i\|\\widehat\{\\Delta\}\_\{\\mathrm\{GDP\}\}\(f\):=n^\{\-1\}\\sum\_\{i=1\}^\{n\}\\lvert\(\\widehat\{m\}\_\{f\}\)\_\{i\}\\rvertsatisfies

\(8\)Δ^GDP​\(f\)2≤1n​∑i=1n\(m^f\)i2≤‖f‖ℱ𝒵2λ^m​HSIC^n​\(Z,S\)\.\\widehat\{\\Delta\}\_\{\\mathrm\{GDP\}\}\(f\)^\{2\}\\;\\leq\\;\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(\\widehat\{m\}\_\{f\}\)\_\{i\}^\{2\}\\;\\leq\\;\\frac\{\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\}\{\\hat\{\\lambda\}\_\{m\}\}\\,\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(Z,S\)\.

###### Proof 4\.13\.

See Appendix[A\.11](https://arxiv.org/html/2608.10470#A1.SS11)\.

#### Interpretation

Equation \([8](https://arxiv.org/html/2608.10470#S4.E8)\) is the term\-by\-term empirical form of Corollary[4\.10](https://arxiv.org/html/2608.10470#S4.Thmtheorem10): the centered Gram matrixL~\\widetilde\{L\}replaces the operatorTST\_\{S\}, itsmm\-th eigenvalueλ^m\\hat\{\\lambda\}\_\{m\}replacesλm\\lambda\_\{m\}, and the projectionm^f=Pm​𝜹f\\widehat\{m\}\_\{f\}=P\_\{m\}\\bm\{\\delta\}\_\{f\}of the prediction onto themmbest\-conditioned sensitive directions replaces the projected conditional meanPm​mfP\_\{m\}m\_\{f\}\. The free parametermmtrades resolution for conditioning, as in the population bound: a largermmresolves more of the prediction’s dependence onSSbut lowersλ^m\\hat\{\\lambda\}\_\{m\}\. A smallHSIC^n\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}therefore boundsΔ^GDP​\(f\)\\widehat\{\\Delta\}\_\{\\mathrm\{GDP\}\}\(f\)for every RKHS headff, through the head\-dependent factor‖f‖ℱ𝒵2/λ^m\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}/\\hat\{\\lambda\}\_\{m\}\.

## 5FRHSIC: A Regularizer with a Faster Train\-to\-Population Rate

FRHSIC turns the population control of the previous section into a trainable penalty whose empirical value bounds the learned encoder’s population dependence at then−1/2n^\{\-1/2\}rate\. It is the empirical objective \([9](https://arxiv.org/html/2608.10470#S5.E9)\) together with its minibatch training algorithm: §[5\.1](https://arxiv.org/html/2608.10470#S5.SS1)defines the objective, and §[5\.2](https://arxiv.org/html/2608.10470#S5.SS2)gives the training algorithm \(the*Implementation*paragraph\) and the uniform train\-to\-population guarantee\. The joint\-distribution framework underlying the HSIC penalty was developed in Section[3](https://arxiv.org/html/2608.10470#S3); this section is self\-contained for a reader who wants only the method and its statistical guarantee\.

### 5\.1Empirical FRHSIC objective

FRHSIC trains the representationZ:=h​\(X\)Z:=h\(X\)\(as in §[2](https://arxiv.org/html/2608.10470#S2)\) by penalizing empirical dependence betweenZZandSSover a feasible encoder classℋ\\mathcal\{H\}and prediction\-head classℱ\\mathcal\{F\}\. With fixed bounded characteristic kernelsk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}, the empirical HSIC penaltyHSIC^n​\(h​\(X\),S\)\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\(X\),S\)is the biasedVV\-statistic \([7](https://arxiv.org/html/2608.10470#S4.E7)\) formed from the paired sample\{\(h​\(Xi\),Si\)\}i=1n\\\{\(h\(X\_\{i\}\),S\_\{i\}\)\\\}\_\{i=1\}^\{n\}\. FRHSIC solves the empirical regularized problem

\(9\)minh∈ℋ,f∈ℱ⁡ℒ^n​\(f∘h\)\+λ​HSIC^n​\(h​\(X\),S\),\\min\_\{h\\in\\mathcal\{H\},\\ f\\in\\mathcal\{F\}\}\\widehat\{\\mathcal\{L\}\}\_\{n\}\(f\\circ h\)\+\\lambda\\,\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\(X\),S\),whereℒ^n\\widehat\{\\mathcal\{L\}\}\_\{n\}is the empirical prediction loss \(cross\-entropy for classification, MSE for regression\) andλ\>0\\lambda\>0controls the fairness–accuracy tradeoff\. Equation \([9](https://arxiv.org/html/2608.10470#S5.E9)\) is the Lagrangian relaxation of the empirical constrained problemminh,f⁡ℒ^n​\(f∘h\)\\min\_\{h,f\}\\widehat\{\\mathcal\{L\}\}\_\{n\}\(f\\circ h\)subject toHSIC^n​\(h​\(X\),S\)≤δ\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\(X\),S\)\\leq\\delta\. We use the biased estimator because it is nonnegative and hence a valid penalty; the unbiasedUU\-statistic estimator\(Song et al\.,[2012](https://arxiv.org/html/2608.10470#bib.bib34)\)can be negative in finite samples, though both are consistent\.

The penalty is a joint\-sample statistic on\{\(h​\(Xi\),Si\)\}\\\{\(h\(X\_\{i\}\),S\_\{i\}\)\\\}and does not construct local estimates ofPZ∣S=sP\_\{Z\\mid S=s\}\. The concentration result below is stated for fixed kernels\. In the experiments the kernel bandwidths are set by the median heuristic and the resulting statistic is used as a practical training penalty; the theorem isolates the fixed\-kernel statistical behavior\.

### 5\.2Uniform train\-to\-population control

Empirical HSIC concentrates on its population value uniformly over the encoder classℋ\\mathcal\{H\}, so the data\-dependent encoder that minimizes the FRHSIC penalty still has small population dependence, up to a uniform error of ordern−1/2n^\{\-1/2\}under controlled encoder complexity\. Uniformity is essential because the penalty is evaluated on an encoder selected from the same sample, so a pointwise concentration bound for a fixed encoder would not suffice\.

###### Theorem 5\.1\(Uniform concentration of empirical HSIC over the encoder class\)\.

Let\{\(Xi,Si\)\}i=1n\\\{\(X\_\{i\},S\_\{i\}\)\\\}\_\{i=1\}^\{n\}be i\.i\.d\. copies of\(X,S\)\(X,S\), and letℋ\\mathcal\{H\}be a class of encodersh:𝒳→𝒵⊆ℝd𝒵h:\\mathcal\{X\}\\to\\mathcal\{Z\}\\subseteq\\mathbb\{R\}^\{d\_\{\\mathcal\{Z\}\}\}\. Forh∈ℋh\\in\\mathcal\{H\}writeZh:=h​\(X\)Z^\{h\}:=h\(X\)andZih:=h​\(Xi\)Z\_\{i\}^\{h\}:=h\(X\_\{i\}\)\. Letk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}be fixed kernels with feature mapsϕ𝒵​\(z\):=k𝒵​\(z,⋅\)∈ℱ𝒵\\phi\_\{\\mathcal\{Z\}\}\(z\):=k\_\{\\mathcal\{Z\}\}\(z,\\cdot\)\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}andϕ𝒮​\(s\):=k𝒮​\(s,⋅\)∈ℱ𝒮\\phi\_\{\\mathcal\{S\}\}\(s\):=k\_\{\\mathcal\{S\}\}\(s,\\cdot\)\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\}, satisfying the boundedness and Lipschitz conditions

supz∈𝒵k𝒵​\(z,z\)≤ν𝒵,sups∈𝒮k𝒮​\(s,s\)≤ν𝒮,\\sup\_\{z\\in\\mathcal\{Z\}\}k\_\{\\mathcal\{Z\}\}\(z,z\)\\leq\\nu\_\{\\mathcal\{Z\}\},\\qquad\\sup\_\{s\\in\\mathcal\{S\}\}k\_\{\\mathcal\{S\}\}\(s,s\)\\leq\\nu\_\{\\mathcal\{S\}\},‖ϕ𝒵​\(z\)−ϕ𝒵​\(z′\)‖ℱ𝒵≤ℓ𝒵​‖z−z′‖,‖ϕ𝒮​\(s\)−ϕ𝒮​\(s′\)‖ℱ𝒮≤ℓ𝒮​‖s−s′‖,\\\|\\phi\_\{\\mathcal\{Z\}\}\(z\)\-\\phi\_\{\\mathcal\{Z\}\}\(z^\{\\prime\}\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\leq\\ell\_\{\\mathcal\{Z\}\}\\,\\\|z\-z^\{\\prime\}\\\|,\\qquad\\\|\\phi\_\{\\mathcal\{S\}\}\(s\)\-\\phi\_\{\\mathcal\{S\}\}\(s^\{\\prime\}\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}\\leq\\ell\_\{\\mathcal\{S\}\}\\,\\\|s\-s^\{\\prime\}\\\|,wherek𝒵​\(z,z\)=‖ϕ𝒵​\(z\)‖ℱ𝒵2k\_\{\\mathcal\{Z\}\}\(z,z\)=\\\|\\phi\_\{\\mathcal\{Z\}\}\(z\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}andk𝒮​\(s,s\)=‖ϕ𝒮​\(s\)‖ℱ𝒮2k\_\{\\mathcal\{S\}\}\(s,s\)=\\\|\\phi\_\{\\mathcal\{S\}\}\(s\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\. Forh∈ℋh\\in\\mathcal\{H\}, writeHSIC^n​\(h\)\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\)for the biased empirical HSIC estimator \([7](https://arxiv.org/html/2608.10470#S4.E7)\) computed from the paired sample\{\(Zih,Si\)\}i=1n\\\{\(Z\_\{i\}^\{h\},S\_\{i\}\)\\\}\_\{i=1\}^\{n\}, andHSIC​\(h\)\\mathrm\{HSIC\}\(h\)for the population HSIC of Definition[2\.1](https://arxiv.org/html/2608.10470#S2.Thmtheorem1)withZ=ZhZ=Z^\{h\}; these are the estimator and functional already defined, evaluated at the encoder\-transformed representationZhZ^\{h\}with the fixed kernelsk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}unchanged\. Let

𝒢^n​\(ℋ\):=𝔼ξ​\[suph∈ℋ1n​∑i=1n⟨ξi,h​\(Xi\)⟩\|X1,…,Xn\],ξi∼iid𝒩​\(0,Id𝒵\),\\widehat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\):=\\mathbb\{E\}\_\{\\xi\}\\\!\\Bigl\[\\sup\_\{h\\in\\mathcal\{H\}\}\\tfrac\{1\}\{n\}\\textstyle\\sum\_\{i=1\}^\{n\}\\langle\\xi\_\{i\},h\(X\_\{i\}\)\\rangle\\,\\Big\|\\,X\_\{1\},\\ldots,X\_\{n\}\\Bigr\],\\quad\\xi\_\{i\}\\stackrel\{\{\\scriptstyle\\mathrm\{iid\}\}\}\{\{\\sim\}\}\\mathcal\{N\}\(0,I\_\{d\_\{\\mathcal\{Z\}\}\}\),be the empirical Gaussian complexity ofℋ\\mathcal\{H\}, and set𝒢n​\(ℋ\):=𝔼​\[𝒢^n​\(ℋ\)\]\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\):=\\mathbb\{E\}\[\\widehat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\)\]\. Define

\(10\)Bn​\(ℋ,δ\):=8​2​ν𝒵​ν𝒮​log⁡\(2/δ\)n\+4​ν𝒵​ν𝒮n\+48​π​max⁡\{ν𝒵​ℓ𝒮,ν𝒮​ℓ𝒵\}​𝒢n​\(ℋ\)\.B\_\{n\}\(\\mathcal\{H\},\\delta\):=8\\sqrt\{2\}\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\\sqrt\{\\tfrac\{\\log\(2/\\delta\)\}\{n\}\}\+\\frac\{4\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\}\{n\}\+48\\sqrt\{\\pi\}\\,\\max\\\{\\nu\_\{\\mathcal\{Z\}\}\\ell\_\{\\mathcal\{S\}\},\\nu\_\{\\mathcal\{S\}\}\\ell\_\{\\mathcal\{Z\}\}\\\}\\,\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)\.Then, for everyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

\(11\)suph∈ℋ\|HSIC^n​\(h\)−HSIC​\(h\)\|≤Bn​\(ℋ,δ\)\.\\sup\_\{h\\in\\mathcal\{H\}\}\\bigl\|\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\)\-\\mathrm\{HSIC\}\(h\)\\bigr\|\\leq B\_\{n\}\(\\mathcal\{H\},\\delta\)\.

###### Proof 5\.2\.

See Appendix[G\.2](https://arxiv.org/html/2608.10470#A7.SS2)\. The proof maps FRHSIC to the uniform HSIC concentration theorem ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\): for eachhh, the empirical HSIC \([7](https://arxiv.org/html/2608.10470#S4.E7)\) at\{\(Zih,Si\)\}i=1n\\\{\(Z\_\{i\}^\{h\},S\_\{i\}\)\\\}\_\{i=1\}^\{n\}is the biased estimator for the product kernelκh​\(\(x,s\),\(x′,s′\)\):=k𝒵​\(h​\(x\),h​\(x′\)\)​k𝒮​\(s,s′\)\\kappa\_\{h\}\\bigl\(\(x,s\),\(x^\{\\prime\},s^\{\\prime\}\)\\bigr\):=k\_\{\\mathcal\{Z\}\}\(h\(x\),h\(x^\{\\prime\}\)\)\\,k\_\{\\mathcal\{S\}\}\(s,s^\{\\prime\}\); the sensitive coordinate is untransformed, so the singleton class on𝒮\\mathcal\{S\}contributes zero Gaussian complexity; and the cited result applied to\{κh:h∈ℋ\}\\\{\\kappa\_\{h\}:h\\in\\mathcal\{H\}\\\}yields \([11](https://arxiv.org/html/2608.10470#S5.E11)\) with the constants of \([10](https://arxiv.org/html/2608.10470#S5.E10)\) \(Proposition[G\.3](https://arxiv.org/html/2608.10470#A7.Thmtheorem3)\)\.

###### Corollary 5\.3\(Population HSIC control for the learned encoder\)\.

Suppose the assumptions of Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1)hold, and leth^∈ℋ\\widehat\{h\}\\in\\mathcal\{H\}be any data\-dependent encoder selected from the same sample, including one obtained by minimizing the FRHSIC objective \([9](https://arxiv.org/html/2608.10470#S5.E9)\)\. Then, with probability at least1−δ1\-\\delta,

HSIC​\(h^\)≤HSIC^n​\(h^\)\+Bn​\(ℋ,δ\)\.\\mathrm\{HSIC\}\(\\widehat\{h\}\)\\leq\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(\\widehat\{h\}\)\+B\_\{n\}\(\\mathcal\{H\},\\delta\)\.

###### Proof 5\.4\.

The event in Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1)holds simultaneously for everyh∈ℋh\\in\\mathcal\{H\}\. Sinceh^∈ℋ\\widehat\{h\}\\in\\mathcal\{H\}, evaluating \([11](https://arxiv.org/html/2608.10470#S5.E11)\) ath=h^h=\\widehat\{h\}gives the claim\.

Corollary[5\.3](https://arxiv.org/html/2608.10470#S5.Thmtheorem3)is the train\-to\-population bridge: small empirical HSIC for the selected encoder implies small population dependence of the learned representation, up to the uniform concentration error, independently of the stochastic optimization used to minimize \([9](https://arxiv.org/html/2608.10470#S5.E9)\)\.

###### Corollary 5\.5\(Root\-nnrate under controlled encoder complexity\)\.

Under the assumptions of Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1), suppose𝒢n​\(ℋ\)=O​\(n−1/2\)\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)=O\(n^\{\-1/2\}\)\. Then, for every fixedδ∈\(0,1\)\\delta\\in\(0,1\),

suph∈ℋ\|HSIC^n​\(h\)−HSIC​\(h\)\|=O~p​\(n−1/2\),\\sup\_\{h\\in\\mathcal\{H\}\}\\bigl\|\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\)\-\\mathrm\{HSIC\}\(h\)\\bigr\|=\\widetilde\{O\}\_\{p\}\(n^\{\-1/2\}\),whereO~\\widetilde\{O\}hides logarithmic factors, and the same rate controls the learned encoderh^\\widehat\{h\}through Corollary[5\.3](https://arxiv.org/html/2608.10470#S5.Thmtheorem3)\.

###### Proof 5\.6\.

Substituting𝒢n​\(ℋ\)=O​\(n−1/2\)\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)=O\(n^\{\-1/2\}\)into \([10](https://arxiv.org/html/2608.10470#S5.E10)\) givesBn​\(ℋ,δ\)=O~​\(n−1/2\)B\_\{n\}\(\\mathcal\{H\},\\delta\)=\\widetilde\{O\}\(n^\{\-1/2\}\); the claim then follows from \([11](https://arxiv.org/html/2608.10470#S5.E11)\) and Corollary[5\.3](https://arxiv.org/html/2608.10470#S5.Thmtheorem3)\.

#### Encoder classes covered by the rate

The condition𝒢n​\(ℋ\)=O​\(n−1/2\)\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)=O\(n^\{\-1/2\}\)is a fixed\-effective\-complexity condition on the feasible encoder class\. For bounded linear encodersℋlin=\{hW​\(x\)=W​x:‖W‖F≤R\}\\mathcal\{H\}\_\{\\mathrm\{lin\}\}=\\\{h\_\{W\}\(x\)=Wx:\\\|W\\\|\_\{F\}\\leq R\\\}with‖X‖≤M\\\|X\\\|\\leq Malmost surely,

𝒢^n​\(ℋlin\)≤R​‖1n​∑i=1nξi​Xi⊤‖F,hence𝒢n​\(ℋlin\)≤R​M​d𝒵/n,\\widehat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\_\{\\mathrm\{lin\}\}\)\\leq R\\,\\bigl\\\|\\tfrac\{1\}\{n\}\\textstyle\\sum\_\{i=1\}^\{n\}\\xi\_\{i\}X\_\{i\}^\{\\top\}\\bigr\\\|\_\{F\},\\qquad\\text\{hence\}\\qquad\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\_\{\\mathrm\{lin\}\}\)\\leq RM\\sqrt\{d\_\{\\mathcal\{Z\}\}/n\},which isO​\(n−1/2\)O\(n^\{\-1/2\}\)for fixed representation dimensiond𝒵d\_\{\\mathcal\{Z\}\}\. The samen−1/2n^\{\-1/2\}order holds for fixed\-depth, fixed\-width MLP encoders with bounded weights, bounded inputs, and Lipschitz activations, and for finite\-dimensional parametric families over compact parameter sets, under the corresponding complexity bounds ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)\. The rate does not extend to arbitrary unregularized or growing\-width networks\.

#### Implementation

Training optimizes \([9](https://arxiv.org/html/2608.10470#S5.E9)\) with minibatch stochastic gradient methods: on a minibatch of sizemmthe HSIC penalty is the same trace formula \([7](https://arxiv.org/html/2608.10470#S4.E7)\) restricted to the batch, atO​\(m2\)O\(m^\{2\}\)cost\. Kernel bandwidths use the median heuristic\. The penalty extends to Equal Opportunity by restricting the statistic to theY=1Y=1subset and to multiple sensitive attributes through a product kernel on the sensitive space, andλ\\lambdais chosen by held\-out validation; details are in Appendix[E\.2](https://arxiv.org/html/2608.10470#A5.SS2)and Appendix[H\.6](https://arxiv.org/html/2608.10470#A8.SS6)\.

## 6Experimental Validation

The experiments are consistent with the theory: the empirical estimator follows the predictedO​\(n−1/2\)O\(n^\{\-1/2\}\)rate, FRHSIC attains fairness–accuracy tradeoffs comparable to conditional\-route baselines while keeping fairness stable across fresh downstream heads, and it trains about36×36\\timesfaster than FREM per epoch\. We evaluate FRHSIC along four questions\. §[6\.2](https://arxiv.org/html/2608.10470#S6.SS2)checks whether the empirical estimator follows the predicted convergence rate\. §[6\.3](https://arxiv.org/html/2608.10470#S6.SS3)compares its fairness–accuracy tradeoffs against the baselines on real data\. §[6\.4](https://arxiv.org/html/2608.10470#S6.SS4)tests whether the enforced fairness transfers to fresh downstream prediction heads\. §[6\.5](https://arxiv.org/html/2608.10470#S6.SS5)measures training time relative to conditional\-route methods\.

Code, cached results, and figures are available in a public repository at[https://github\.com/Yijin911/FRHSIC](https://github.com/Yijin911/FRHSIC), with an archived DOI release accompanying the camera\-ready version\.

### 6\.1Experimental setup

We use five real datasets with continuous sensitive attributes\. Adult, ACS Income, MEPS, and COMPAS are classification tasks scored by accuracy; Crime is a regression task scored by mean squared error\. Fairness is measured by held\-outΔGDP\\Delta\_\{\\mathrm\{GDP\}\}, with lower values indicating less dependence on the sensitive attributeSS\.

- •Adult:n≈30,000n\\approx 30\{,\}000individuals; income\>50\{\>\}50K; sensitive attribute age\.
- •ACS Income:n=20,000n=20\{,\}000from the 2018 California American Community Survey via folktables\(Ding et al\.,[2021](https://arxiv.org/html/2608.10470#bib.bib9)\); income\>50\{\>\}50K; age\.
- •MEPS:n≈13,000n\\approx 13\{,\}000from the 2015 Medical Expenditure Panel Survey\(Romano et al\.,[2020](https://arxiv.org/html/2608.10470#bib.bib31)\); healthcare utilization≥10\\geq 10visits; age\.
- •Crime\(the Communities and Crime dataset\):n≈2,000n\\approx 2\{,\}000communities; violent crimes per population; racial composition\.
- •COMPAS:n≈6,000n\\approx 6\{,\}000defendants; two\-year recidivism; age\.

Features andSSare min–max scaled to\[0,1\]\[0,1\], fit on the training split only\. We compare FRHSIC with the continuous\-sensitive methods FREM\(Kong et al\.,[2025](https://arxiv.org/html/2608.10470#bib.bib18)\)and Reg\-GDP\(Jiang et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib15)\), the adversarial method ADV\(Grari et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib11)\), the binary\-attribute methods MMD\(Deka and Sutherland,[2023](https://arxiv.org/html/2608.10470#bib.bib8)\)and LAFTR\(Madras et al\.,[2018](https://arxiv.org/html/2608.10470#bib.bib24)\)applied to quartile\-binnedSS, and an unconstrained baseline \(Unfair,λ=0\\lambda=0\)\. FollowingKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\), all methods share a two\-layer SELU encoder with hidden and representation dimension5050and a linear prediction head, train for200200epochs with Adam at learning rate10−310^\{\-3\}, and are swept overλ∈\{0\.1,1,10,100,500\}\\lambda\\in\\\{0\.1,1,10,100,500\\\}with results averaged over five random80/2080/20splits, reported as mean±\\pmstandard deviation\. Each method keeps the kernel bandwidth convention of its source publication; we do not retune bandwidths to favor any method, and the only fairness\-specific hyperparameter FRHSIC tunes isλ\\lambda, with kernel bandwidths fixed by the median heuristic;λ\\lambdais selected by the held\-out procedure of Appendix[H\.6](https://arxiv.org/html/2608.10470#A8.SS6)\. Full preprocessing, hyperparameter grids, hardware, and the FREM3232\-anchor subsampling approximation are documented in Appendix[D](https://arxiv.org/html/2608.10470#A4)\.

### 6\.2Synthetic estimator behavior

Empirical HSIC converges to its population value at the predictedO​\(n−1/2\)O\(n^\{\-1/2\}\)rate, faster than the conditional\-route EIPM estimator’sO​\(n−2/5\)O\(n^\{\-2/5\}\)\. We draw\(Z,S\)\(Z,S\)jointly Gaussian with correlationρ=0\.5\\rho=0\.5\(Z∼𝒩​\(0,1\)Z\\sim\\mathcal\{N\}\(0,1\),S=ρ​Z\+1−ρ2​εS=\\rho Z\+\\sqrt\{1\-\\rho^\{2\}\}\\,\\varepsilon,ε∼𝒩​\(0,1\)\\varepsilon\\sim\\mathcal\{N\}\(0,1\)\), approximate each estimator’s population value by itsn=20,000n=20\{,\}000estimate, and report the absolute error at sample sizes from5050to5,0005\{,\}000, averaged over2020replications with Gaussian kernels at bandwidth11\. The EIPM estimator is the weighted MMD estimator ofKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\), the same estimator used by the FREM baseline\. On this dependence,HSIC^\\widehat\{\\mathrm\{HSIC\}\}converges at theO​\(n−1/2\)O\(n^\{\-1/2\}\)rate that underlies the uniform train\-to\-population control of §[5\.2](https://arxiv.org/html/2608.10470#S5.SS2), while the EIPM estimator converges at the slowerO​\(n−2/5\)O\(n^\{\-2/5\}\)rate\. Figure[2](https://arxiv.org/html/2608.10470#S6.F2)reports both estimation errors against sample size\. By Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8), HSIC and the conditional MMD integral bound each other up to an explicit spectral tail, so the joint estimator’s fasterO​\(n−1/2\)O\(n^\{\-1/2\}\)rate is a statistical\-efficiency gain in estimating a closely related fairness quantity, equal to the conditional MMD integral up to that tail\. The conditional\-gap bound of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)is separately validated on a synthetic classification setup in Appendix[E\.1](https://arxiv.org/html/2608.10470#A5.SS1)\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x2.png)Figure 2:Synthetic estimator convergence\. Absolute estimation error is plotted against sample size on log–log axes\. Empirical HSIC follows the predictedO​\(n−1/2\)O\(n^\{\-1/2\}\)behavior and the smoothing\-based EIPM estimator follows the slowerO​\(n−2/5\)O\(n^\{\-2/5\}\)trend: the fitted log–log slopes are−0\.46\-0\.46for HSIC and−0\.44\-0\.44for EIPM, close to the predicted−1/2\-1/2and−2/5\-2/5\. The dashed and dotted gray lines are then−1/2n^\{\-1/2\}andn−2/5n^\{\-2/5\}references\.
### 6\.3Fairness–accuracy tradeoffs

On the five real datasets of §[6\.1](https://arxiv.org/html/2608.10470#S6.SS1), scored by held\-outΔGDP\\Delta\_\{\\mathrm\{GDP\}\}against accuracy \(mean squared error for Crime\), FRHSIC achieves fairness–accuracy tradeoffs comparable to the strongest continuous\-sensitive baselines, although it is not uniformly best on any dataset\. Because HSIC and GDP penalties have different scales, equalλ\\lambdadoes not mean equal fairness, so Table[1](https://arxiv.org/html/2608.10470#S6.T1)compares at a matched operating point: for each method we report the lowestΔGDP\\Delta\_\{\\mathrm\{GDP\}\}among models whose prediction performance stays within1%1\\%of the Unfair baseline, together with the bandwidth\-free normalized area under the accuracy–fairness Pareto envelope, nAUP\. Figures[3](https://arxiv.org/html/2608.10470#S6.F3)and[4](https://arxiv.org/html/2608.10470#S6.F4)show the full frontiers over theλ\\lambdasweep, for the classification datasets and for Crime respectively\.

No method dominates across datasets\. By nAUP, FRHSIC is best on COMPAS, within\.004\.004of the best on Crime, and mid\-pack on Adult, ACS Income, and MEPS, staying within the spread of FREM, Reg\-GDP, and the adversarial baselines\. Among the binned baselines, LAFTR is consistently among the weakest by nAUP, while MMD on quartile\-binnedSSis competitive on Adult and ACS Income; binningSSthus yields no systematic advantage over the continuous\-sensitive methods on these datasets\. The matched band also removes a Reg\-GDP artifact: Reg\-GDP reaches near\-zeroΔGDP\\Delta\_\{\\mathrm\{GDP\}\}only by collapsing accuracy outside the1%1\\%band, from\.849\.849to\.758\.758on Adult and from\.790\.790to\.590\.590on ACS Income at its unconstrained point\. Differences within±0\.01\\pm 0\.01accuracy or±0\.05\\pm 0\.05onΔGDP\\Delta\_\{\\mathrm\{GDP\}\}fall inside the five\-split spread and should be read as practically equivalent\. COMPAS age is integer\-valued with few distinct values, so its centered sensitive Gram matrix has small rank, and the finite\-sample bound of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)is governed by a few well\-conditioned sensitive directions\.

Table 1:Matched\-performance comparison on the five real datasets\. For each method we report the lowestΔGDP\\Delta\_\{\\mathrm\{GDP\}\}among models whose performance stays within1%1\\%of the Unfair baseline\. Perf is accuracy, except Crime where lowerMSE×102\\mathrm\{MSE\}\\times 10^\{2\}is better;ΔGDP\\Delta\_\{\\mathrm\{GDP\}\}lower is fairer; nAUP is the normalized area under the accuracy–fairness Pareto envelope, higher is better\. Bold marks the best fairness method per column; “—” means no sweptλ\\lambdameets the band\. Setup and bandwidth conventions are in §[6\.1](https://arxiv.org/html/2608.10470#S6.SS1)\.DatasetMethodPerf\.ΔGDP\\Delta\_\{\\mathrm\{GDP\}\}\(↓\\downarrow\)nAUP \(↑\\uparrow\)Adult \(Acc↑\\uparrow\)Unfair\.849\.849\.749\.749—FRHSIC \(Ours\)\.845\.845\.232\.232\.786\.786FREM\.845\.845\.167\.167\.835\.835Reg\-GDP\.842\.842\.082\\mathbf\{\.082\}\.872\.872ADV\.841\.841\.466\.466\.434\.434MMD \(binned\)\.847\.847\.126\.126\.880\\mathbf\{\.880\}LAFTR \(binned\)\.842\.842\.385\.385\.555\.555ACS Income \(Acc↑\\uparrow\)Unfair\.790\.790\.514\.514—FRHSIC \(Ours\)\.783\.783\.228\.228\.805\.805FREM\.787\.787\.274\.274\.865\.865Reg\-GDP\.787\.787\.292\.292\.852\.852ADV\.785\.785\.458\.458\.713\.713MMD \(binned\)\.783\.783\.157\\mathbf\{\.157\}\.881\\mathbf\{\.881\}LAFTR \(binned\)\.787\.787\.474\.474\.754\.754MEPS \(Acc↑\\uparrow\)Unfair\.804\.804\.626\.626—FRHSIC \(Ours\)\.797\.797\.329\.329\.464\.464FREM\.800\.800\.317\\mathbf\{\.317\}\.549\\mathbf\{\.549\}Reg\-GDP\.801\.801\.370\.370\.469\.469ADV\.803\.803\.480\.480\.487\.487MMD \(binned\)\.800\.800\.407\.407\.492\.492LAFTR \(binned\)\.803\.803\.530\.530\.477\.477Crime \(MSE↓\\downarrow\)Unfair1\.911\.91\.029\.029—FRHSIC \(Ours\)1\.911\.91\.029\.029\.686\.686FREM1\.871\.87\.023\\mathbf\{\.023\}\.680\.680Reg\-GDP1\.911\.91\.025\.025\.690\\mathbf\{\.690\}ADV1\.921\.92\.029\.029\.002\.002MMD \(binned\)——\.573\.573LAFTR \(binned\)1\.891\.89\.029\.029\.103\.103COMPAS \(Acc↑\\uparrow\)Unfair\.654\.654\.123\.123—FRHSIC \(Ours\)\.649\.649\.090\.090\.679\\mathbf\{\.679\}FREM\.648\.648\.101\.101\.530\.530Reg\-GDP\.650\.650\.079\\mathbf\{\.079\}\.480\.480ADV\.649\.649\.095\.095\.314\.314MMD \(binned\)\.651\.651\.082\.082\.485\.485LAFTR \(binned\)\.655\.655\.122\.122\.475\.475![Refer to caption](https://arxiv.org/html/2608.10470v1/x3.png)Figure 3:Fairness–accuracy frontiers on the classification datasets\. Each point corresponds to one value of the fairness regularization parameterλ\\lambda; lowerΔGDP\\Delta\_\{\\mathrm\{GDP\}\}is fairer and higher accuracy is better\. Lines connect the Pareto\-efficient points of each method\. Axes are scaled separately by dataset to show the frontier structure\. The black star marks the Unfair baseline, the unconstrained model trained without any fairness penalty \(λ=0\\lambda=0\)\. FRHSIC is competitive with continuous\-sensitive baselines across datasets, but no method uniformly dominates\. Table[1](https://arxiv.org/html/2608.10470#S6.T1)gives the matched\-performance summary\.![Refer to caption](https://arxiv.org/html/2608.10470v1/x4.png)Figure 4:Fairness–prediction\-error frontier on Crime\. Each point corresponds to one value ofλ\\lambda; lowerΔGDP\\Delta\_\{\\mathrm\{GDP\}\}is fairer and lower MSE is better\. Lines connect the Pareto\-efficient points of each method; the black star marks the Unfair baseline \(λ=0\\lambda=0, no fairness penalty\)\. The separate panel avoids mixing regression error with classification accuracy\.
### 6\.4Transfer to fresh downstream heads

Fairness enforced by FRHSIC stays stable across fresh downstream heads, consistent with a penalty that constrains all dependence betweenZZandSSrather than the single head used in training\. On Adult \(atλ=10\\lambda=10\) and Crime \(atλ=100\\lambda=100\), we freeze the learned representation of each method and train four fresh heads on it: linear, two\-layer MLP, random forest, and SVM\. Table[2](https://arxiv.org/html/2608.10470#S6.T2)reports the standard deviation ofΔGDP\\Delta\_\{\\mathrm\{GDP\}\}across the four heads, a stability diagnostic in which lower values indicate fairness less tied to the head used during training\. FRHSIC’s cross\-head variability is comparable to FREM and, on Adult, well below the Unfair baseline\. Reg\-GDP attains lower variability on Adult, but only at the accuracy collapse noted in §[6\.3](https://arxiv.org/html/2608.10470#S6.SS3)\. This is a stability diagnostic rather than a guarantee that every downstream head is fair; the full per\-head breakdown is in Appendix[H\.2](https://arxiv.org/html/2608.10470#A8.SS2)\.

Table 2:Cross\-headΔGDP\\Delta\_\{\\mathrm\{GDP\}\}standard deviation across four fresh heads \(linear, MLP, random forest, SVM\) trained on a single frozen representation atλ=10\\lambda=10\(Adult\) andλ=100\\lambda=100\(Crime\); lower indicates fairness less tied to the training head\. Full per\-head results are in Appendix[H\.2](https://arxiv.org/html/2608.10470#A8.SS2)\.MethodAdult \(λ=10\\lambda=10\)Crime \(λ=100\\lambda=100\)Unfair0\.3480\.348—FRHSIC \(Ours\)0\.2440\.2440\.0010\.001FREM0\.2470\.2470\.0020\.002Reg\-GDP0\.0580\.058†0\.0020\.002ADV0\.2520\.2520\.0050\.005
†Reg\-GDP attains lower cross\-head variance on Adult but with accuracy collapsing to0\.7580\.758\(Table[1](https://arxiv.org/html/2608.10470#S6.T1)\)\.

### 6\.5Computational efficiency

FRHSIC trains substantially faster than FREM because it replaces FREM’s per\-sample weighted conditional\-MMD computation with a singleO​\(m2\)O\(m^\{2\}\)HSIC trace per minibatch\. Figure[5](https://arxiv.org/html/2608.10470#S6.F5)reports per\-epoch wall\-clock time, averaged over five epochs after a warm\-up epoch, on a synthetic binary\-classification task \(S∼Unif​\(0,1\)S\\sim\\mathrm\{Unif\}\(0,1\), featuresX=\(S\+ε,ε′\)X=\(S\+\\varepsilon,\\,\\varepsilon^\{\\prime\}\)withε∼𝒩​\(0,0\.32\)\\varepsilon\\sim\\mathcal\{N\}\(0,0\.3^\{2\}\)andε′∼𝒩​\(0,1\)\\varepsilon^\{\\prime\}\\sim\\mathcal\{N\}\(0,1\), labelY∼Bernoulli​\(σ​\(X1\)\)Y\\sim\\mathrm\{Bernoulli\}\(\\sigma\(X\_\{1\}\)\)\) of varying size, at batch size256256fromn=500n=500ton=20,000n=20\{,\}000, with all methods sharing the encoder and prediction head\. Atn=20,000n=20\{,\}000and batch size256256, FRHSIC completes an epoch in2\.822\.82s versus100\.7100\.7s for FREM, a roughly36×36\\timesspeedup\. Reg\-GDP is faster still at1\.691\.69s because it evaluates conditional expectations on a fixed grid rather than kernel matrices, but it optimizes a head\-specific smoothed moment criterion, whereas FRHSIC penalizes a representation\-level distributional dependence that binds every downstream head\. The full\-batchO​\(n2\)O\(n^\{2\}\)versusO​\(n3\)O\(n^\{3\}\)analysis is given in Appendix[D](https://arxiv.org/html/2608.10470#A4)\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x5.png)Figure 5:Per\-epoch wall\-clock time versus sample size at batch size256256, on log–log axes; error bars are±1\\pm 1standard deviation over repeats and are negligible except for FRHSIC at the smallestnn\. FRHSIC uses one HSIC trace per batch and trains about36×36\\timesfaster than FREM atn=20,000n=20\{,\}000\(§[6\.5](https://arxiv.org/html/2608.10470#S6.SS5)\)\. Reg\-GDP is faster but optimizes a head\-specific smoothed criterion rather than a representation\-level distributional penalty\.Additional experiments are reported in Appendices[H\.2](https://arxiv.org/html/2608.10470#A8.SS2)–[H\.6](https://arxiv.org/html/2608.10470#A8.SS6): an empirical tightness sweep for the Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)bound, a high\-dimensional sensitive\-attribute comparison, a mutual\-information diagnostic, ablations over kernel choice and representation dimensionality, and validation of theλ\\lambda\-selection rule of Appendix[H\.6](https://arxiv.org/html/2608.10470#A8.SS6)\.

## 7Related Work

#### Fair representation learning

Zemel et al\. \([2013](https://arxiv.org/html/2608.10470#bib.bib38)\)introduced the framework of mapping data to a latent space that preserves task information while obfuscatingSS\. Subsequent work extended this through variational autoencoders\(Louizos et al\.,[2016](https://arxiv.org/html/2608.10470#bib.bib21)\), adversarial training\(Madras et al\.,[2018](https://arxiv.org/html/2608.10470#bib.bib24)\), disentangled representations\(Creager et al\.,[2019](https://arxiv.org/html/2608.10470#bib.bib7)\), MMD\- and Sinkhorn\-based formulations\(Cho et al\.,[2020a](https://arxiv.org/html/2608.10470#bib.bib4); Deka and Sutherland,[2023](https://arxiv.org/html/2608.10470#bib.bib8)\), and data\-domain transformations\(Quadrianto et al\.,[2019](https://arxiv.org/html/2608.10470#bib.bib30)\)\. Most of these methods are designed for categorical sensitive attributes and discretize continuousSSat training time\.

#### Continuous\-sensitive\-attribute fairness criteria

GDP\(Jiang et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib15)\)is estimated via Nadaraya–Watson kernel smoothing onSS\. EIPM/FREM\(Kong et al\.,[2025](https://arxiv.org/html/2608.10470#bib.bib18)\)extends GDP to higher moments via a weighted empirical MMD with bandwidthγn\\gamma\_\{n\}and leave\-one\-out correction\. Kernel HGR / NLD\(Mary et al\.,[2019](https://arxiv.org/html/2608.10470#bib.bib25); Grari et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib11)\)estimates a global dependence criterion via regularized kernel canonical correlation\. Mutual information\(Cho et al\.,[2020b](https://arxiv.org/html/2608.10470#bib.bib5)\)is estimated via density or copula approximations\. Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)of §[3\.2](https://arxiv.org/html/2608.10470#S3.SS2)disintegrates the joint\-vs\-product IPM into aPSP\_\{S\}\-weighted conditional contrast, and Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)recovers the conditional\-integral IPM that GDP and EIPM instantiate when the witness class decomposes acrossss; kHGR is a supremum\-of\-correlations criterion outside this form\.

#### HSIC and dependence measures

HSIC was introduced byGretton et al\. \([2005](https://arxiv.org/html/2608.10470#bib.bib12)\)and has been used for independence testing\(Gretton et al\.,[2007](https://arxiv.org/html/2608.10470#bib.bib13); Albert et al\.,[2022](https://arxiv.org/html/2608.10470#bib.bib1)\), feature selection\(Song et al\.,[2007](https://arxiv.org/html/2608.10470#bib.bib33),[2012](https://arxiv.org/html/2608.10470#bib.bib34)\), dimensionality reduction\(Ma et al\.,[2018](https://arxiv.org/html/2608.10470#bib.bib23)\), and as a training objective for deep networks\(Ma et al\.,[2020](https://arxiv.org/html/2608.10470#bib.bib22)\); its equivalence with distance covariance\(Sejdinovic et al\.,[2013](https://arxiv.org/html/2608.10470#bib.bib32)\)places it within a single family of joint\-distribution dependence measures\. HSIC has also been used directly as a fairness penalty:Pérez\-Suay et al\. \([2017](https://arxiv.org/html/2608.10470#bib.bib29)\)introduced it as the fairness term in kernel regression and dimensionality reduction,Li et al\. \([2022](https://arxiv.org/html/2608.10470#bib.bib20)\)added closed\-form ridge solutions and a Gaussian\-process treatment, andQuadrianto et al\. \([2019](https://arxiv.org/html/2608.10470#bib.bib30)\)used it for independence from the sensitive attribute in a data\-domain model\. Our contribution is the continuous\-SSdisintegration that ties the HSIC penalty to the conditional\-integral criteria \(Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3), Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)\) and the spectral and train\-to\-population guarantees \(Theorems[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)and[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1)\) that these empirical and ridge\-based treatments lack\.

## 8Discussion

#### The joint\-distribution view

The joint\-distribution view reaches the same independence target as the conditional\-integral criteria but changes the object that is estimated\. Rather than a conditional law at each sensitive value, it estimates a single closed\-formVV\-statistic, which controls smoothed conditional deviations and, under spectral regularity, the conditional MMD integral up to a spectral tail \(Theorems[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4)and[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\)\.

#### Extensions

The framework extends to representation\-level Equal Opportunity by computing the joint discrepancy on theY=1Y=1subset, and to multiple continuous sensitive attributes by taking a product kernel on𝒮1×𝒮2\\mathcal\{S\}\_\{1\}\\times\\mathcal\{S\}\_\{2\}\. Both extensions are derived in Appendix[E\.2](https://arxiv.org/html/2608.10470#A5.SS2)\. Replacing the reference distributionPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}byPZ⊗Unif​\(𝒮\)P\_\{Z\}\\otimes\\mathrm\{Unif\}\(\\mathcal\{S\}\)yields a minority\-protection target in which rare values ofSSreceive equal weight; this is a modeling choice and does not affect the structural identity of Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)\.

#### Open questions

Two questions remain\. First, the empirical bound in Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)scales as‖f‖ℱZ2/λ^m\\\|f\\\|\_\{\\mathcal\{F\}\_\{Z\}\}^\{2\}/\\hat\{\\lambda\}\_\{m\}; tighter spectral arguments exploiting the full singular\-value distribution ofL~\\widetilde\{L\}would yield sharper guarantees on the integral form\. Second, the framework treats the joint discrepancy with an RKHS witness class; non\-RKHS instances \(Wasserstein, total variation\) inherit the structural identity but lose theO​\(n−1/2\)O\(n^\{\-1/2\}\)rate and closed\-form estimator\. We leave these to future work\.

## Appendix AProofs

This section collects the proofs of the results stated in the main text, in the order in which the results appear there\. The technical lemma used in the proof of Proposition[4\.1](https://arxiv.org/html/2608.10470#S4.Thmtheorem1)\(Lemma[B\.1](https://arxiv.org/html/2608.10470#A2.Thmtheorem1)\) is recorded in Appendix[B](https://arxiv.org/html/2608.10470#A2)below\.

### A\.1Proof of Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1): conditions \(1\)–\(4\)

###### Proof A\.1\.

We show\(i\)⇒\(i​i\)⇒\(i​i​i\)⇒\(i​v\)⇒\(i\)\(i\)\\Rightarrow\(ii\)\\Rightarrow\(iii\)\\Rightarrow\(iv\)\\Rightarrow\(i\)\.

*\(i\)⇒\(i​i\)\(i\)\\Rightarrow\(ii\)\.*Sinceℱ\\mathcal\{F\}is characteristic,IPMℱ​\(PZ,S,PZ⊗PS\)=0\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\)=0impliesPZ,S=PZ⊗PSP\_\{Z,S\}=P\_\{Z\}\\otimes P\_\{S\}\.

*\(i​i\)⇒\(i​i​i\)\(ii\)\\Rightarrow\(iii\)\.*On the Polish product space𝒵×𝒮\\mathcal\{Z\}\\times\\mathcal\{S\}, the disintegration theorem\(Kallenberg,[2002](https://arxiv.org/html/2608.10470#bib.bib16), Thm\. 5\.4\)yields a regular conditional probability\{PZ\|S=s\}s∈𝒮\\\{P\_\{Z\|S=s\}\\\}\_\{s\\in\\mathcal\{S\}\}such that for every BorelA⊆𝒵,B⊆𝒮A\\subseteq\\mathcal\{Z\},B\\subseteq\\mathcal\{S\},

PZ,S​\(A×B\)=∫BPZ\|S=s​\(A\)​𝑑PS​\(s\)\.P\_\{Z,S\}\(A\\times B\)=\\int\_\{B\}P\_\{Z\|S=s\}\(A\)\\,dP\_\{S\}\(s\)\.Under\(i​i\)\(ii\), the same Borel setA×BA\\times Balso satisfiesPZ,S​\(A×B\)=PZ​\(A\)​PS​\(B\)=∫BPZ​\(A\)​𝑑PS​\(s\)P\_\{Z,S\}\(A\\times B\)=P\_\{Z\}\(A\)\\,P\_\{S\}\(B\)=\\int\_\{B\}P\_\{Z\}\(A\)\\,dP\_\{S\}\(s\)\. Comparing the two expressions,∫B\[PZ\|S=s​\(A\)−PZ​\(A\)\]​𝑑PS​\(s\)=0\\int\_\{B\}\[P\_\{Z\|S=s\}\(A\)\-P\_\{Z\}\(A\)\]\\,dP\_\{S\}\(s\)=0for every BorelBB, soPZ\|S=s​\(A\)=PZ​\(A\)P\_\{Z\|S=s\}\(A\)=P\_\{Z\}\(A\)forPSP\_\{S\}\-a\.e\.ss\. Choose a countable algebra𝒜\\mathcal\{A\}generating the Borelσ\\sigma\-algebra of𝒵\\mathcal\{Z\}\(which exists since𝒵\\mathcal\{Z\}is Polish\); the exceptionalPSP\_\{S\}\-null sets, taken over the countable family𝒜\\mathcal\{A\}, have countable union ofPSP\_\{S\}\-measure zero\. HencePZ\|S=s\|𝒜=PZ\|𝒜P\_\{Z\|S=s\}\|\_\{\\mathcal\{A\}\}=P\_\{Z\}\|\_\{\\mathcal\{A\}\}forPSP\_\{S\}\-a\.e\.ss, and by theπ\\pi\-λ\\lambdatheoremPZ\|S=s=PZP\_\{Z\|S=s\}=P\_\{Z\}on the full Borelσ\\sigma\-algebra forPSP\_\{S\}\-a\.e\.ss\. Equivalence of “some” and “every” regular conditional version follows from thePSP\_\{S\}\-a\.e\. uniqueness clause of the disintegration theorem\.

*\(i​i​i\)⇒\(i​v\)\(iii\)\\Rightarrow\(iv\)\.*For every bounded measurableffandPSP\_\{S\}\-a\.e\.ss,

𝔼​\[f​\(Z\)∣S=s\]=∫f​\(z\)​𝑑PZ\|S=s​\(z\)=∫f​\(z\)​𝑑PZ​\(z\)=𝔼​\[f​\(Z\)\],\\mathbb\{E\}\[f\(Z\)\\mid S=s\]=\\int f\(z\)\\,dP\_\{Z\|S=s\}\(z\)=\\int f\(z\)\\,dP\_\{Z\}\(z\)=\\mathbb\{E\}\[f\(Z\)\],where the second equality uses\(i​i​i\)\(iii\)\.

*\(i​v\)⇒\(i\)\(iv\)\\Rightarrow\(i\)\.*Indicator functions of Borel sets are bounded and measurable, so applying\(i​v\)\(iv\)withf=𝟏Af=\\mathbf\{1\}\_\{A\}givesPZ\|S=s​\(A\)=PZ​\(A\)P\_\{Z\|S=s\}\(A\)=P\_\{Z\}\(A\)forPSP\_\{S\}\-a\.e\.ss, for every BorelA⊆𝒵A\\subseteq\\mathcal\{Z\}\. Take a countable algebra𝒜\\mathcal\{A\}generating the Borelσ\\sigma\-algebra of the Polish space𝒵\\mathcal\{Z\}; the exceptionalPSP\_\{S\}\-null sets over the countable family𝒜\\mathcal\{A\}have null union, soPZ\|S=s\|𝒜=PZ\|𝒜P\_\{Z\|S=s\}\|\_\{\\mathcal\{A\}\}=P\_\{Z\}\|\_\{\\mathcal\{A\}\}forPSP\_\{S\}\-a\.e\.ss, and theπ\\pi\-λ\\lambdatheorem givesPZ\|S=s=PZP\_\{Z\|S=s\}=P\_\{Z\}forPSP\_\{S\}\-a\.e\.ss\. HenceZ⟂SZ\\perp S, i\.e\.,PZ,S=PZ⊗PSP\_\{Z,S\}=P\_\{Z\}\\otimes P\_\{S\}, soIPMℱ​\(PZ,S,PZ⊗PS\)=0\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\)=0\.

### A\.2Proof of Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1): the MMD\-based EIPM instance

###### Proof A\.2\.

\(⇒\\Rightarrow\) IfHSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0, thenμZ,S=μZ⊗μS\\mu\_\{Z,S\}=\\mu\_\{Z\}\\otimes\\mu\_\{S\}inℱ𝒵⊗𝒮\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\. Becausek𝒵k\_\{\\mathcal\{Z\}\}andk𝒮k\_\{\\mathcal\{S\}\}are characteristic, the product kernelk𝒵⊗k𝒮k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}isℐ\\mathcal\{I\}\-characteristic, i\.e\., its joint mean embedding distinguishesPZ,SP\_\{Z,S\}fromPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}\(Theorem 3 ofSzabó and Sriperumbudur,[2018](https://arxiv.org/html/2608.10470#bib.bib35), takingM=2M=2\)\. HenceμZ,S=μZ⊗μS\\mu\_\{Z,S\}=\\mu\_\{Z\}\\otimes\\mu\_\{S\}impliesPZ,S=PZ⊗PSP\_\{Z,S\}=P\_\{Z\}\\otimes P\_\{S\}, i\.e\.,Z⟂SZ\\perp S\. HencePZ∣S=s=PZP\_\{Z\\mid S=s\}=P\_\{Z\}forPSP\_\{S\}\-almost everyss, soMMD​\(PZ∣S=s,PZ\)=0\\mathrm\{MMD\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)=0a\.s\., andEIPM𝒱​\(Z;S\)=𝔼S​\[MMD​\(PZ∣S,PZ\)\]=0\\mathrm\{EIPM\}\_\{\\mathcal\{V\}\}\(Z;S\)=\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]=0\.

\(⇐\\Leftarrow\) SupposeEIPM𝒱​\(Z;S\)=0\\mathrm\{EIPM\}\_\{\\mathcal\{V\}\}\(Z;S\)=0\. SinceMMD​\(PZ∣S=s,PZ\)≥0\\mathrm\{MMD\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\\geq 0, this impliesMMD​\(PZ∣S=s,PZ\)=0\\mathrm\{MMD\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)=0forPSP\_\{S\}\-almost everyss\. Becausek𝒵k\_\{\\mathcal\{Z\}\}is characteristic, MMD is a metric on probability distributions, soPZ∣S=s=PZP\_\{Z\\mid S=s\}=P\_\{Z\}a\.s\., i\.e\.,Z⟂SZ\\perp S\. Independence then implies that the joint mean embedding factorizes,μZ,S=μZ⊗μS\\mu\_\{Z,S\}=\\mu\_\{Z\}\\otimes\\mu\_\{S\}, soHSIC​\(Z,S\)=‖μZ,S−μZ⊗μS‖ℱ𝒵⊗𝒮2=0\\mathrm\{HSIC\}\(Z,S\)=\\\|\\mu\_\{Z,S\}\-\\mu\_\{Z\}\\otimes\\mu\_\{S\}\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}=0\.

### A\.3Proof of Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1): downstream\-head \(GDP\) form

Sincek𝒵⊗k𝒮k\_\{\\mathcal\{Z\}\}\\otimes k\_\{\\mathcal\{S\}\}is characteristic,HSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0is the product\-RKHS instance of condition \(1\) of Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1), soPZ∣S=s=PZP\_\{Z\\mid S=s\}=P\_\{Z\}forPSP\_\{S\}\-almost everyssby condition \(3\), hence𝔼​\[f​\(Z\)∣S=s\]=𝔼​\[f​\(Z\)\]\\mathbb\{E\}\[f\(Z\)\\mid S=s\]=\\mathbb\{E\}\[f\(Z\)\]forPSP\_\{S\}\-almost everyssand every measurablef:𝒵→ℝf:\\mathcal\{Z\}\\to\\mathbb\{R\}with𝔼​\|f​\(Z\)\|<∞\\mathbb\{E\}\|f\(Z\)\|<\\infty\. ThereforeΔGDP\(f\)=𝔼S\|𝔼\[f\(Z\)∣S\]−𝔼\[f\(Z\)\]\|=0\\Delta\_\{\\mathrm\{GDP\}\}\(f\)=\\mathbb\{E\}\_\{S\}\\bigl\|\\mathbb\{E\}\[f\(Z\)\\mid S\]\-\\mathbb\{E\}\[f\(Z\)\]\\bigr\|=0\.

### A\.4Proof of Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3)\(Disintegration of the joint\-vs\-product IPM\)

Because𝒵\\mathcal\{Z\}and𝒮\\mathcal\{S\}are standard Borel, the disintegration theorem\(Kallenberg,[2002](https://arxiv.org/html/2608.10470#bib.bib16), Theorem 5\.4\)givesPZ,S​\(d​z​d​s\)=PZ∣S=s​\(d​z\)​PS​\(d​s\)P\_\{Z,S\}\(dz\\,ds\)=P\_\{Z\\mid S=s\}\(dz\)\\,P\_\{S\}\(ds\), and by construction\(PZ⊗PS\)​\(d​z​d​s\)=PZ​\(d​z\)​PS​\(d​s\)\(P\_\{Z\}\\otimes P\_\{S\}\)\(dz\\,ds\)=P\_\{Z\}\(dz\)\\,P\_\{S\}\(ds\)\. Fixf∈ℱf\\in\\mathcal\{F\}with‖f‖∞≤b<∞\\\|f\\\|\_\{\\infty\}\\leq b<\\infty\. BothPZ∣S=s​\(d​z\)​PS​\(d​s\)P\_\{Z\\mid S=s\}\(dz\)\\,P\_\{S\}\(ds\)andPZ​\(d​z\)​PS​\(d​s\)P\_\{Z\}\(dz\)\\,P\_\{S\}\(ds\)are probability measures and\|f\|≤b\|f\|\\leq b, so each iterated integral below is absolutely convergent and the Fubini–Tonelli theorem applies:

𝔼PZ,S​\[f\]−𝔼PZ⊗PS​\[f\]=∫𝒮\(∫𝒵f​\(z,s\)​PZ∣S=s​\(d​z\)−∫𝒵f​\(z,s\)​PZ​\(d​z\)\)​PS​\(d​s\)\.\\mathbb\{E\}\_\{P\_\{Z,S\}\}\[f\]\-\\mathbb\{E\}\_\{P\_\{Z\}\\otimes P\_\{S\}\}\[f\]=\\int\_\{\\mathcal\{S\}\}\\\!\\left\(\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,P\_\{Z\\mid S=s\}\(dz\)\-\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,P\_\{Z\}\(dz\)\\right\)P\_\{S\}\(ds\)\.The inner difference is∫𝒵f​\(z,s\)​d​\(PZ∣S=s−PZ\)​\(z\)\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,d\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\)\(z\)\. Taking absolute values and the supremum overf∈ℱf\\in\\mathcal\{F\}yields the identity\.

### A\.5Proof of Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)\(Recovery of the conditional\-integral IPM\)

By Theorem[3\.3](https://arxiv.org/html/2608.10470#S3.Thmtheorem3),

IPMℱ​\(PZ,S,PZ⊗PS\)=supf∈ℱ\|𝔼S​∫𝒵f​\(z,S\)​d​\(PZ∣S−PZ\)​\(z\)\|\.\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\(P\_\{Z,S\},P\_\{Z\}\\otimes P\_\{S\}\)=\\sup\_\{f\\in\\mathcal\{F\}\}\\Bigl\|\\,\\mathbb\{E\}\_\{S\}\\\!\\int\_\{\\mathcal\{Z\}\}f\(z,S\)\\,d\(P\_\{Z\\mid S\}\-P\_\{Z\}\)\(z\)\\Bigr\|\.*Upper bound\.*Fixf∈ℱf\\in\\mathcal\{F\}\. By Definition[3\.5](https://arxiv.org/html/2608.10470#S3.Thmtheorem5)\(2\),f​\(⋅,s\)∈𝒱f\(\\cdot,s\)\\in\\mathcal\{V\}forPSP\_\{S\}\-a\.e\.ss, so for thosess

\|∫𝒵f​\(z,s\)​d​\(PZ∣S=s−PZ\)​\(z\)\|≤supg∈𝒱\|∫𝒵g​d​\(PZ∣S=s−PZ\)\|=d𝒱​\(PZ∣S=s,PZ\)\.\\Bigl\|\\int\_\{\\mathcal\{Z\}\}f\(z,s\)\\,d\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\)\(z\)\\Bigr\|\\leq\\sup\_\{g\\in\\mathcal\{V\}\}\\Bigl\|\\int\_\{\\mathcal\{Z\}\}g\\,d\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\)\\Bigr\|=d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\.By the triangle inequality for thePSP\_\{S\}\-integral,\|𝔼S​∫f​\(z,S\)​d​\(PZ∣S−PZ\)\|≤𝔼S​\[d𝒱​\(PZ∣S,PZ\)\]\\bigl\|\\mathbb\{E\}\_\{S\}\\\!\\int f\(z,S\)\\,d\(P\_\{Z\\mid S\}\-P\_\{Z\}\)\\bigr\|\\leq\\mathbb\{E\}\_\{S\}\\bigl\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\\bigr\]; taking the supremum overf∈ℱf\\in\\mathcal\{F\}givesIPMℱ≤𝔼S​\[d𝒱​\(PZ∣S,PZ\)\]\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\leq\\mathbb\{E\}\_\{S\}\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]\.*Lower bound\.*Fixε\>0\\varepsilon\>0\. Since𝒱\\mathcal\{V\}is symmetric \(g∈𝒱⇒−g∈𝒱g\\in\\mathcal\{V\}\\Rightarrow\-g\\in\\mathcal\{V\}\),d𝒱​\(PZ∣S=s,PZ\)=supg∈𝒱\(∫g​𝑑PZ∣S=s−∫g​𝑑PZ\)d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)=\\sup\_\{g\\in\\mathcal\{V\}\}\\bigl\(\\int g\\,dP\_\{Z\\mid S=s\}\-\\int g\\,dP\_\{Z\}\\bigr\), the absolute value being attained because−g∈𝒱\-g\\in\\mathcal\{V\}\. By the selector assumption of Corollary[3\.6](https://arxiv.org/html/2608.10470#S3.Thmtheorem6)there is a measurable maps↦gsε∈𝒱s\\mapsto g\_\{s\}^\{\\varepsilon\}\\in\\mathcal\{V\}with∫gsε​d​\(PZ∣S=s−PZ\)≥d𝒱​\(PZ∣S=s,PZ\)−ε\\int g\_\{s\}^\{\\varepsilon\}\\,d\(P\_\{Z\\mid S=s\}\-P\_\{Z\}\)\\geq d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\-\\varepsilonforPSP\_\{S\}\-a\.e\.ss\. By Definition[3\.5](https://arxiv.org/html/2608.10470#S3.Thmtheorem5)\(1\),f⋆​\(z,s\):=gsε​\(z\)f\_\{\\star\}\(z,s\):=g\_\{s\}^\{\\varepsilon\}\(z\)lies inℱ\\mathcal\{F\}, and since the integrand is nonnegative up toε\\varepsilon,

IPMℱ≥\|𝔼S​∫gSε​d​\(PZ∣S−PZ\)\|≥𝔼S​\[d𝒱​\(PZ∣S,PZ\)\]−ε\.\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\geq\\Bigl\|\\mathbb\{E\}\_\{S\}\\\!\\int g\_\{S\}^\{\\varepsilon\}\\,d\(P\_\{Z\\mid S\}\-P\_\{Z\}\)\\Bigr\|\\geq\\mathbb\{E\}\_\{S\}\\bigl\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\\bigr\]\-\\varepsilon\.Lettingε↓0\\varepsilon\\downarrow 0givesIPMℱ≥𝔼S​\[d𝒱​\(PZ∣S,PZ\)\]\\mathrm\{IPM\}\_\{\\mathcal\{F\}\}\\geq\\mathbb\{E\}\_\{S\}\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]\. Combining the two bounds and recallingℐd𝒱​\(Z;S\)=𝔼S​\[d𝒱​\(PZ∣S,PZ\)\]\\mathcal\{I\}\_\{d\_\{\\mathcal\{V\}\}\}\(Z;S\)=\\mathbb\{E\}\_\{S\}\[d\_\{\\mathcal\{V\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]from the preliminaries yields the identity\.

### A\.6Proof of Proposition[4\.1](https://arxiv.org/html/2608.10470#S4.Thmtheorem1)\(HSIC and Expected MMD\)

###### Proof A\.3\.

Step 1: Decomposition\.By definition of the joint mean embedding and the tower property of conditional expectation,

μZ,S=𝔼Z,S​\[k𝒵​\(Z,⋅\)⊗k𝒮​\(S,⋅\)\]=𝔼S​\[μZ∣S⊗k𝒮​\(S,⋅\)\],\\mu\_\{Z,S\}=\\mathbb\{E\}\_\{Z,S\}\\\!\\left\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\right\]=\\mathbb\{E\}\_\{S\}\\\!\\left\[\\mu\_\{Z\\mid S\}\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\right\],and by the definitionμS=𝔼S′​\[k𝒮​\(S′,⋅\)\]\\mu\_\{S\}=\\mathbb\{E\}\_\{S^\{\\prime\}\}\[k\_\{\\mathcal\{S\}\}\(S^\{\\prime\},\\cdot\)\]and linearity of the tensor product \(withμZ\\mu\_\{Z\}constant inS′S^\{\\prime\}\),μZ⊗μS=𝔼S′​\[μZ⊗k𝒮​\(S′,⋅\)\]\.\\mu\_\{Z\}\\otimes\\mu\_\{S\}=\\mathbb\{E\}\_\{S^\{\\prime\}\}\[\\mu\_\{Z\}\\otimes k\_\{\\mathcal\{S\}\}\(S^\{\\prime\},\\cdot\)\]\.Subtracting and using linearity of the Bochner integral \(Lemma[B\.1](https://arxiv.org/html/2608.10470#A2.Thmtheorem1)\),

\(12\)μZ,S−μZ⊗μS=𝔼S​\[\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)\]\.\\mu\_\{Z,S\}\-\\mu\_\{Z\}\\otimes\\mu\_\{S\}=\\mathbb\{E\}\_\{S\}\\\!\\left\[\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\right\]\.Squaring the norm and writing it as an inner product gives, withDS:=\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)D\_\{S\}:=\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\),

HSIC​\(Z,S\)\\displaystyle\\mathrm\{HSIC\}\(Z,S\)=⟨𝔼S​\[DS\],𝔼S′​\[DS′\]⟩ℱ𝒵⊗𝒮\\displaystyle=\\langle\\mathbb\{E\}\_\{S\}\[D\_\{S\}\],\\,\\mathbb\{E\}\_\{S^\{\\prime\}\}\[D\_\{S^\{\\prime\}\}\]\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}=𝔼S,S′​\[⟨DS,DS′⟩ℱ𝒵⊗𝒮\],\\displaystyle=\\mathbb\{E\}\_\{S,S^\{\\prime\}\}\\\!\\left\[\\langle D\_\{S\},\\,D\_\{S^\{\\prime\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}\\right\],where the exchange of expectation and inner product is justified by Lemma[B\.1](https://arxiv.org/html/2608.10470#A2.Thmtheorem1)together with the bound‖\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)‖ℱ𝒵⊗𝒮≤2​κ𝒵​κ𝒮\\\|\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}\\leq 2\\sqrt\{\\kappa\_\{\\mathcal\{Z\}\}\\kappa\_\{\\mathcal\{S\}\}\}\. By the tensor\-product inner product identity⟨a⊗b,a′⊗b′⟩=⟨a,a′⟩​⟨b,b′⟩\\langle a\\otimes b,a^\{\\prime\}\\otimes b^\{\\prime\}\\rangle=\\langle a,a^\{\\prime\}\\rangle\\langle b,b^\{\\prime\}\\rangleand the reproducing property⟨k𝒮​\(S,⋅\),k𝒮​\(S′,⋅\)⟩ℱ𝒮=k𝒮​\(S,S′\)\\langle k\_\{\\mathcal\{S\}\}\(S,\\cdot\),k\_\{\\mathcal\{S\}\}\(S^\{\\prime\},\\cdot\)\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}=k\_\{\\mathcal\{S\}\}\(S,S^\{\\prime\}\),

HSIC​\(Z,S\)=𝔼S,S′​\[⟨μZ∣S−μZ,μZ∣S′−μZ⟩ℱ𝒵⋅k𝒮​\(S,S′\)\],\\mathrm\{HSIC\}\(Z,S\)=\\mathbb\{E\}\_\{S,S^\{\\prime\}\}\\\!\\left\[\\langle\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\},\\mu\_\{Z\\mid S^\{\\prime\}\}\-\\mu\_\{Z\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\cdot k\_\{\\mathcal\{S\}\}\(S,S^\{\\prime\}\)\\right\],which establishes \([4](https://arxiv.org/html/2608.10470#S4.E4)\)\.

Step 2: Squared expected MMD bound\.Starting from \([12](https://arxiv.org/html/2608.10470#A1.E12)\), apply the triangle inequality \(Jensen’s inequality for the convex norm\):

HSIC​\(Z,S\)1/2\\displaystyle\\mathrm\{HSIC\}\(Z,S\)^\{1/2\}=‖𝔼S​\[\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)\]‖ℱ𝒵⊗𝒮\\displaystyle=\\left\\\|\\mathbb\{E\}\_\{S\}\[\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\right\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}≤𝔼S​\[‖\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)‖ℱ𝒵⊗𝒮\]\.\\displaystyle\\leq\\mathbb\{E\}\_\{S\}\\\!\\left\[\\\|\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}\\right\]\.The tensor\-product norm factorizes:‖a⊗b‖ℱ𝒵⊗𝒮=‖a‖ℱ𝒵​‖b‖ℱ𝒮\\\|a\\otimes b\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}=\\\|a\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\\|b\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}, and‖k𝒮​\(S,⋅\)‖ℱ𝒮=k𝒮​\(S,S\)≤κ𝒮\\\|k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}=\\sqrt\{k\_\{\\mathcal\{S\}\}\(S,S\)\}\\leq\\sqrt\{\\kappa\_\{\\mathcal\{S\}\}\}\. Therefore

HSIC​\(Z,S\)1/2≤𝔼S​\[‖μZ∣S−μZ‖ℱ𝒵​k𝒮​\(S,S\)\]≤κ𝒮⋅𝔼S​\[MMDk𝒵​\(PZ∣S,PZ\)\]\.\\mathrm\{HSIC\}\(Z,S\)^\{1/2\}\\leq\\mathbb\{E\}\_\{S\}\\\!\\left\[\\\|\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\sqrt\{k\_\{\\mathcal\{S\}\}\(S,S\)\}\\right\]\\leq\\sqrt\{\\kappa\_\{\\mathcal\{S\}\}\}\\cdot\\mathbb\{E\}\_\{S\}\\\!\\left\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\\right\]\.Squaring both sides yields \([5](https://arxiv.org/html/2608.10470#S4.E5)\)\.

### A\.7Proof of Theorem[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4)\(Smoothed conditional\-distribution control by HSIC\)

###### Proof A\.4\.

Sincek𝒵k\_\{\\mathcal\{Z\}\}is bounded,‖μZ∣S=s‖ℱ𝒵≤κ𝒵\\\|\\mu\_\{Z\\mid S=s\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\leq\\sqrt\{\\kappa\_\{\\mathcal\{Z\}\}\}and‖μZ‖ℱ𝒵≤κ𝒵\\\|\\mu\_\{Z\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\leq\\sqrt\{\\kappa\_\{\\mathcal\{Z\}\}\}, soΔ∈L2​\(PS;ℱ𝒵\)\\Delta\\in L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\. The joint mean embedding satisfies

μZ,S−μZ⊗μS=𝔼S​\[\(μZ∣S=S−μZ\)⊗k𝒮​\(S,⋅\)\]=𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]\.\\mu\_\{Z,S\}\-\\mu\_\{Z\}\\otimes\\mu\_\{S\}=\\mathbb\{E\}\_\{S\}\[\(\\mu\_\{Z\\mid S=S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]=\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\.Taking the squared norm inℱ𝒵⊗𝒮\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}gives the first identity\.

Forg∈ℱ𝒮g\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\}, the partial contraction of the tensor againstggover theℱ𝒮\\mathcal\{F\}\_\{\\mathcal\{S\}\}factor gives an element ofℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}:

\(idℱ𝒵⊗⟨⋅,g⟩ℱ𝒮\)​\(𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]\)=𝔼S​\[g​\(S\)​Δ​\(S\)\]\.\\bigl\(\\mathrm\{id\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\otimes\\langle\\,\\cdot\\,,g\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}\\bigr\)\\bigl\(\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\bigr\)=\\mathbb\{E\}\_\{S\}\[g\(S\)\\,\\Delta\(S\)\]\.Hence

‖𝔼S​\[g​\(S\)​Δ​\(S\)\]‖ℱ𝒵≤‖g‖ℱ𝒮​‖𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]‖ℱ𝒵⊗𝒮,\\bigl\\\|\\mathbb\{E\}\_\{S\}\[g\(S\)\\Delta\(S\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\leq\\\|g\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}\\,\\bigl\\\|\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\},which, with the first identity, proves \([6](https://arxiv.org/html/2608.10470#S4.E6)\)\. Two equivalent reformulations follow\. Writingmf​\(s\):=𝔼​\[f​\(Z\)∣S=s\]−𝔼​\[f​\(Z\)\]=⟨f,Δ​\(s\)⟩ℱ𝒵m\_\{f\}\(s\):=\\mathbb\{E\}\[f\(Z\)\\mid S=s\]\-\\mathbb\{E\}\[f\(Z\)\]=\\langle f,\\Delta\(s\)\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}forf∈ℱ𝒵f\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}, the same contraction argument applied to theℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\-coordinate gives the fixed\-head form

\(13\)‖𝔼S​\[mf​\(S\)​k𝒮​\(S,⋅\)\]‖ℱ𝒮2≤‖f‖ℱ𝒵2​HSIC​\(Z,S\)\.\\bigl\\\|\\mathbb\{E\}\_\{S\}\[m\_\{f\}\(S\)\\,k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\\leq\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\,\\mathrm\{HSIC\}\(Z,S\)\.Finally, since𝔼S​\[g​\(S\)​mf​\(S\)\]=⟨f,𝔼S​\[g​\(S\)​Δ​\(S\)\]⟩ℱ𝒵\\mathbb\{E\}\_\{S\}\[g\(S\)m\_\{f\}\(S\)\]=\\bigl\\langle f,\\;\\mathbb\{E\}\_\{S\}\[g\(S\)\\Delta\(S\)\]\\bigr\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}, Cauchy–Schwarz together with \([6](https://arxiv.org/html/2608.10470#S4.E6)\) gives the bilinear form

\(14\)\|𝔼S​\[g​\(S\)​mf​\(S\)\]\|2≤‖f‖ℱ𝒵2​‖g‖ℱ𝒮2​HSIC​\(Z,S\)\\bigl\|\\mathbb\{E\}\_\{S\}\[g\(S\)m\_\{f\}\(S\)\]\\bigr\|^\{2\}\\leq\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\,\\\|g\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\\,\\mathrm\{HSIC\}\(Z,S\)for allf∈ℱ𝒵f\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}andg∈ℱ𝒮g\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\}\.

### A\.8Proof of Corollary[4\.6](https://arxiv.org/html/2608.10470#S4.Thmtheorem6)\(Localized sensitive\-value contrasts\)

###### Proof A\.5\.

Apply Inequality \([6](https://arxiv.org/html/2608.10470#S4.E6)\) withg=k𝒮​\(⋅,s0\)∈ℱ𝒮g=k\_\{\\mathcal\{S\}\}\(\\cdot,s\_\{0\}\)\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\}: by the reproducing propertyg​\(S\)=k𝒮​\(S,s0\)g\(S\)=k\_\{\\mathcal\{S\}\}\(S,s\_\{0\}\)and‖g‖ℱ𝒮2=⟨k𝒮​\(⋅,s0\),k𝒮​\(⋅,s0\)⟩ℱ𝒮=k𝒮​\(s0,s0\)\\\|g\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}=\\langle k\_\{\\mathcal\{S\}\}\(\\cdot,s\_\{0\}\),k\_\{\\mathcal\{S\}\}\(\\cdot,s\_\{0\}\)\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}=k\_\{\\mathcal\{S\}\}\(s\_\{0\},s\_\{0\}\), so‖𝔼S​\[k𝒮​\(S,s0\)​Δ​\(S\)\]‖ℱ𝒵2≤k𝒮​\(s0,s0\)​HSIC​\(Z,S\)\\bigl\\\|\\mathbb\{E\}\_\{S\}\[k\_\{\\mathcal\{S\}\}\(S,s\_\{0\}\)\\Delta\(S\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq k\_\{\\mathcal\{S\}\}\(s\_\{0\},s\_\{0\}\)\\,\\mathrm\{HSIC\}\(Z,S\), which is the single\-point bound sinceΔ​\(S\)=μZ∣S−μZ\\Delta\(S\)=\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\. The two\-point form follows by the same argument withg=k𝒮​\(⋅,s0\)−k𝒮​\(⋅,s1\)∈ℱ𝒮g=k\_\{\\mathcal\{S\}\}\(\\cdot,s\_\{0\}\)\-k\_\{\\mathcal\{S\}\}\(\\cdot,s\_\{1\}\)\\in\\mathcal\{F\}\_\{\\mathcal\{S\}\}, for whichg​\(S\)=k𝒮​\(S,s0\)−k𝒮​\(S,s1\)g\(S\)=k\_\{\\mathcal\{S\}\}\(S,s\_\{0\}\)\-k\_\{\\mathcal\{S\}\}\(S,s\_\{1\}\)and‖g‖ℱ𝒮2=‖k𝒮​\(s0,⋅\)−k𝒮​\(s1,⋅\)‖ℱ𝒮2\\\|g\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}=\\\|k\_\{\\mathcal\{S\}\}\(s\_\{0\},\\cdot\)\-k\_\{\\mathcal\{S\}\}\(s\_\{1\},\\cdot\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{S\}\}\}^\{2\}\.

### A\.9Proof of Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\(upper bound: spectral control of the conditional MMD integral\)

###### Proof A\.6\.

Sincek𝒮k\_\{\\mathcal\{S\}\}is bounded,TST\_\{S\}is a self\-adjoint, positive, trace\-class operator onL2​\(PS\)L^\{2\}\(P\_\{S\}\); let\(λℓ,ψℓ\)ℓ≥1\(\\lambda\_\{\\ell\},\\psi\_\{\\ell\}\)\_\{\\ell\\geq 1\}be its eigensystem withλℓ≥0\\lambda\_\{\\ell\}\\geq 0and\{ψℓ\}\\\{\\psi\_\{\\ell\}\\\}orthonormal inL2​\(PS\)L^\{2\}\(P\_\{S\}\), and recall the Mercer expansionk𝒮​\(s,t\)=∑ℓ≥1λℓ​ψℓ​\(s\)​ψℓ​\(t\)k\_\{\\mathcal\{S\}\}\(s,t\)=\\sum\_\{\\ell\\geq 1\}\\lambda\_\{\\ell\}\\psi\_\{\\ell\}\(s\)\\psi\_\{\\ell\}\(t\)inL2​\(PS⊗PS\)L^\{2\}\(P\_\{S\}\\otimes P\_\{S\}\)\. By Theorem[4\.4](https://arxiv.org/html/2608.10470#S4.Thmtheorem4),

HSIC​\(Z,S\)=‖𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]‖ℱ𝒵⊗𝒮2\.\\mathrm\{HSIC\}\(Z,S\)=\\bigl\\\|\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]\\bigr\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\\otimes\\mathcal\{S\}\}\}^\{2\}\.Substituting the Mercer expansion and writingΔℓ:=𝔼S​\[Δ​\(S\)​ψℓ​\(S\)\]=∫Δ​\(s\)​ψℓ​\(s\)​𝑑PS​\(s\)∈ℱ𝒵\\Delta\_\{\\ell\}:=\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\psi\_\{\\ell\}\(S\)\]=\\int\\Delta\(s\)\\psi\_\{\\ell\}\(s\)\\,dP\_\{S\}\(s\)\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}gives, by orthonormality of\{ψℓ\}\\\{\\psi\_\{\\ell\}\\\},

𝔼S​\[Δ​\(S\)⊗k𝒮​\(S,⋅\)\]=∑ℓ≥1λℓ​Δℓ⊗ψℓ,HSIC​\(Z,S\)=∑ℓ≥1λℓ​‖Δℓ‖ℱ𝒵2,\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]=\\sum\_\{\\ell\\geq 1\}\\lambda\_\{\\ell\}\\,\\Delta\_\{\\ell\}\\otimes\\psi\_\{\\ell\},\\qquad\\mathrm\{HSIC\}\(Z,S\)=\\sum\_\{\\ell\\geq 1\}\\lambda\_\{\\ell\}\\\|\\Delta\_\{\\ell\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\},which is the first identity\. For anymmwithλm\>0\\lambda\_\{m\}\>0, sinceλℓ≥λm\\lambda\_\{\\ell\}\\geq\\lambda\_\{m\}forℓ≤m\\ell\\leq mand every term is nonnegative,

∑ℓ=1m‖Δℓ‖ℱ𝒵2≤1λm​∑ℓ=1mλℓ​‖Δℓ‖ℱ𝒵2≤1λm​HSIC​\(Z,S\)\.\\sum\_\{\\ell=1\}^\{m\}\\\|\\Delta\_\{\\ell\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\frac\{1\}\{\\lambda\_\{m\}\}\\sum\_\{\\ell=1\}^\{m\}\\lambda\_\{\\ell\}\\\|\\Delta\_\{\\ell\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\frac\{1\}\{\\lambda\_\{m\}\}\\mathrm\{HSIC\}\(Z,S\)\.Because\{ψℓ\}\\\{\\psi\_\{\\ell\}\\\}is orthonormal inL2​\(PS\)L^\{2\}\(P\_\{S\}\), theℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\-valued coefficients ofΔ\\DeltaareΔℓ\\Delta\_\{\\ell\}, so‖Pm​Δ‖L2​\(PS;ℱ𝒵\)2=∑ℓ=1m‖Δℓ‖ℱ𝒵2\\\|P\_\{m\}\\Delta\\\|\_\{L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\}^\{2\}=\\sum\_\{\\ell=1\}^\{m\}\\\|\\Delta\_\{\\ell\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}, giving the projected bound\. Finally,PmP\_\{m\}is an orthogonal projection onL2​\(PS;ℱ𝒵\)L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\), so‖Δ‖2=‖Pm​Δ‖2\+‖\(I−Pm\)​Δ‖2≤λm−1​HSIC​\(Z,S\)\+ρm2\\\|\\Delta\\\|^\{2\}=\\\|P\_\{m\}\\Delta\\\|^\{2\}\+\\\|\(I\-P\_\{m\}\)\\Delta\\\|^\{2\}\\leq\\lambda\_\{m\}^\{\-1\}\\mathrm\{HSIC\}\(Z,S\)\+\\rho\_\{m\}^\{2\}; the identity‖Δ‖L2​\(PS;ℱ𝒵\)2=𝔼S​\[‖Δ​\(S\)‖ℱ𝒵2\]=𝔼S​\[MMDk𝒵2​\(PZ∣S,PZ\)\]\\\|\\Delta\\\|\_\{L^\{2\}\(P\_\{S\};\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\)\}^\{2\}=\\mathbb\{E\}\_\{S\}\[\\\|\\Delta\(S\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\]=\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}^\{2\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]uses‖Δ​\(s\)‖ℱ𝒵=MMDk𝒵​\(PZ∣S=s,PZ\)\\\|\\Delta\(s\)\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}=\\mathrm\{MMD\}\_\{k\_\{\\mathcal\{Z\}\}\}\(P\_\{Z\\mid S=s\},P\_\{Z\}\)\.

### A\.10Proof of Corollary[4\.10](https://arxiv.org/html/2608.10470#S4.Thmtheorem10)\(Spectral control of GDP for RKHS heads\)

###### Proof A\.7\.

By the reproducing property,mf​\(s\)=𝔼​\[f​\(Z\)∣S=s\]−𝔼​\[f​\(Z\)\]=⟨f,Δ​\(s\)⟩ℱ𝒵m\_\{f\}\(s\)=\\mathbb\{E\}\[f\(Z\)\\mid S=s\]\-\\mathbb\{E\}\[f\(Z\)\]=\\langle f,\\Delta\(s\)\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}, somf∈L2​\(PS\)m\_\{f\}\\in L^\{2\}\(P\_\{S\}\)withL2​\(PS\)L^\{2\}\(P\_\{S\}\)coefficients∫mf​\(s\)​ψℓ​\(s\)​𝑑PS​\(s\)=⟨f,Δℓ⟩ℱ𝒵\\int m\_\{f\}\(s\)\\psi\_\{\\ell\}\(s\)\\,dP\_\{S\}\(s\)=\\langle f,\\Delta\_\{\\ell\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}, whereΔℓ:=𝔼S​\[Δ​\(S\)​ψℓ​\(S\)\]\\Delta\_\{\\ell\}:=\\mathbb\{E\}\_\{S\}\[\\Delta\(S\)\\psi\_\{\\ell\}\(S\)\]as in the proof of Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8)\. Hence, by Cauchy–Schwarz and that theorem,

‖Pm​mf‖L2​\(PS\)2=∑ℓ=1m⟨f,Δℓ⟩ℱ𝒵2≤‖f‖ℱ𝒵2​∑ℓ=1m‖Δℓ‖ℱ𝒵2≤‖f‖ℱ𝒵2λm​HSIC​\(Z,S\)\.\\\|P\_\{m\}m\_\{f\}\\\|\_\{L^\{2\}\(P\_\{S\}\)\}^\{2\}=\\sum\_\{\\ell=1\}^\{m\}\\langle f,\\Delta\_\{\\ell\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\sum\_\{\\ell=1\}^\{m\}\\\|\\Delta\_\{\\ell\}\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\leq\\frac\{\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\}\{\\lambda\_\{m\}\}\\,\\mathrm\{HSIC\}\(Z,S\)\.SincePmP\_\{m\}is an orthogonal projection onL2​\(PS\)L^\{2\}\(P\_\{S\}\),‖mf‖L2​\(PS\)2=‖Pm​mf‖2\+‖\(I−Pm\)​mf‖2≤λm−1​‖f‖ℱ𝒵2​HSIC​\(Z,S\)\+rm,f2\\\|m\_\{f\}\\\|\_\{L^\{2\}\(P\_\{S\}\)\}^\{2\}=\\\|P\_\{m\}m\_\{f\}\\\|^\{2\}\+\\\|\(I\-P\_\{m\}\)m\_\{f\}\\\|^\{2\}\\leq\\lambda\_\{m\}^\{\-1\}\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\mathrm\{HSIC\}\(Z,S\)\+r\_\{m,f\}^\{2\}\. Finally, by Jensen’s inequalityΔGDP​\(f\)=𝔼S​\|mf​\(S\)\|≤\(𝔼S​mf​\(S\)2\)1/2=‖mf‖L2​\(PS\)\\Delta\_\{\\mathrm\{GDP\}\}\(f\)=\\mathbb\{E\}\_\{S\}\|m\_\{f\}\(S\)\|\\leq\(\\mathbb\{E\}\_\{S\}m\_\{f\}\(S\)^\{2\}\)^\{1/2\}=\\\|m\_\{f\}\\\|\_\{L^\{2\}\(P\_\{S\}\)\}, soΔGDP​\(f\)2≤λm−1​‖f‖ℱ𝒵2​HSIC​\(Z,S\)\+rm,f2\\Delta\_\{\\mathrm\{GDP\}\}\(f\)^\{2\}\\leq\\lambda\_\{m\}^\{\-1\}\\\|f\\\|\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}^\{2\}\\mathrm\{HSIC\}\(Z,S\)\+r\_\{m,f\}^\{2\}\.

### A\.11Proof of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\(Empirical demographic\-parity control by HSIC\)

###### Proof A\.8\(Proof of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\)\.

We work with empirical quantities throughout\. Denote byd^Si:=k𝒵​\(Zi,⋅\)−μ^Z\\hat\{d\}\_\{S\_\{i\}\}:=k\_\{\\mathcal\{Z\}\}\(Z\_\{i\},\\cdot\)\-\\hat\{\\mu\}\_\{Z\}the empirical centered embedding atS=SiS=S\_\{i\}, withμ^Z=n−1​∑j=1nk𝒵​\(Zj,⋅\)\\hat\{\\mu\}\_\{Z\}=n^\{\-1\}\\sum\_\{j=1\}^\{n\}k\_\{\\mathcal\{Z\}\}\(Z\_\{j\},\\cdot\), and write𝛅f:=\(δ^f,1,…,δ^f,n\)⊤\\bm\{\\delta\}\_\{f\}:=\(\\hat\{\\delta\}\_\{f,1\},\\ldots,\\hat\{\\delta\}\_\{f,n\}\)^\{\\top\}for the centered prediction\-deviation vector\.

Step 1: Spectral identity forHSIC^\\widehat\{\\mathrm\{HSIC\}\}using positive eigenpairs only\.LetK∈ℝn×nK\\in\\mathbb\{R\}^\{n\\times n\}have entriesKi​j=k𝒵​\(Zi,Zj\)K\_\{ij\}=k\_\{\\mathcal\{Z\}\}\(Z\_\{i\},Z\_\{j\}\)andK~=H​K​H\\widetilde\{K\}=HKH\. The biased V\-statistic estimator is

HSIC^​\(Z,S\)=1n2​tr​\(K​H​L​H\)=1n2​tr​\(K~​L~\)\.\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)=\\frac\{1\}\{n^\{2\}\}\\mathrm\{tr\}\(KHLH\)=\\frac\{1\}\{n^\{2\}\}\\mathrm\{tr\}\(\\widetilde\{K\}\\,\\widetilde\{L\}\)\.SinceLLis symmetric positive semi\-definite andH=H⊤H=H^\{\\top\}withH2=HH^\{2\}=H,L~=H​L​H\\widetilde\{L\}=HLHis symmetric PSD\. Letλ^1≥⋯≥λ^r\>0\\hat\{\\lambda\}\_\{1\}\\geq\\cdots\\geq\\hat\{\\lambda\}\_\{r\}\>0be the positive eigenvalues ofn−1​L~n^\{\-1\}\\widetilde\{L\}with associated orthonormal eigenvectorse^1,…,e^r∈ℝn\\hat\{e\}\_\{1\},\\ldots,\\hat\{e\}\_\{r\}\\in\\mathbb\{R\}^\{n\}, wherer=rank​\(L~\)r=\\mathrm\{rank\}\(\\widetilde\{L\}\)\. Then

1n​L~=∑j=1rλ^j​e^j​e^j⊤,Pm=∑j=1me^j​e^j⊤\.\\frac\{1\}\{n\}\\widetilde\{L\}=\\sum\_\{j=1\}^\{r\}\\hat\{\\lambda\}\_\{j\}\\,\\hat\{e\}\_\{j\}\\hat\{e\}\_\{j\}^\{\\top\},\\qquad P\_\{m\}=\\sum\_\{j=1\}^\{m\}\\hat\{e\}\_\{j\}\\hat\{e\}\_\{j\}^\{\\top\}\.Substituting and usingHSIC^=n−1​tr​\(K~⋅n−1​L~\)\\widehat\{\\mathrm\{HSIC\}\}=n^\{\-1\}\\mathrm\{tr\}\(\\widetilde\{K\}\\cdot n^\{\-1\}\\widetilde\{L\}\),

\(15\)HSIC^​\(Z,S\)=1n​∑j=1rλ^j⋅e^j⊤​K~​e^j\.\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)=\\frac\{1\}\{n\}\\sum\_\{j=1\}^\{r\}\\hat\{\\lambda\}\_\{j\}\\cdot\\hat\{e\}\_\{j\}^\{\\top\}\\widetilde\{K\}\\hat\{e\}\_\{j\}\.
Step 2: Lower bound via themm\-th eigenvalue\.SinceK~=H​K​H\\widetilde\{K\}=HKHis PSD, every term in \([15](https://arxiv.org/html/2608.10470#A1.E15)\) is nonnegative; dropping the termsj\>mj\>mand usingλ^j≥λ^m\\hat\{\\lambda\}\_\{j\}\\geq\\hat\{\\lambda\}\_\{m\}forj≤mj\\leq m,

\(16\)HSIC^​\(Z,S\)≥1n​∑j=1mλ^j​e^j⊤​K~​e^j≥λ^mn​∑j=1me^j⊤​K~​e^j=λ^mn​tr​\(Pm​K~​Pm\),\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)\\;\\geq\\;\\frac\{1\}\{n\}\\sum\_\{j=1\}^\{m\}\\hat\{\\lambda\}\_\{j\}\\,\\hat\{e\}\_\{j\}^\{\\top\}\\widetilde\{K\}\\hat\{e\}\_\{j\}\\;\\geq\\;\\frac\{\\hat\{\\lambda\}\_\{m\}\}\{n\}\\sum\_\{j=1\}^\{m\}\\hat\{e\}\_\{j\}^\{\\top\}\\widetilde\{K\}\\hat\{e\}\_\{j\}\\;=\\;\\frac\{\\hat\{\\lambda\}\_\{m\}\}\{n\}\\,\\mathrm\{tr\}\\\!\\big\(P\_\{m\}\\,\\widetilde\{K\}\\,P\_\{m\}\\big\),where we used∑j=1me^j⊤​K~​e^j=tr​\(Pm​K~​Pm\)\\sum\_\{j=1\}^\{m\}\\hat\{e\}\_\{j\}^\{\\top\}\\widetilde\{K\}\\hat\{e\}\_\{j\}=\\mathrm\{tr\}\(P\_\{m\}\\widetilde\{K\}P\_\{m\}\)\(sincePm=∑j=1me^j​e^j⊤P\_\{m\}=\\sum\_\{j=1\}^\{m\}\\hat\{e\}\_\{j\}\\hat\{e\}\_\{j\}^\{\\top\},Pm2=PmP\_\{m\}^\{2\}=P\_\{m\}, and the cyclic\-trace identity\)\.

Step 3: Bound the projected prediction deviation bytr​\(Pm​K~​Pm\)\\mathrm\{tr\}\(P\_\{m\}\\widetilde\{K\}P\_\{m\}\)\.By the reproducing property,δ^f,i=⟨f,d^Si⟩ℱ𝒵\\hat\{\\delta\}\_\{f,i\}=\\langle f,\\hat\{d\}\_\{S\_\{i\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}, so the vector𝛅f\\bm\{\\delta\}\_\{f\}has entries\(𝛅f\)i=⟨f,d^Si⟩ℱ𝒵\(\\bm\{\\delta\}\_\{f\}\)\_\{i\}=\\langle f,\\hat\{d\}\_\{S\_\{i\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\. For any unit vectoru∈ℝnu\\in\\mathbb\{R\}^\{n\}, the linear combination∑iui​d^Si∈ℱ𝒵\\sum\_\{i\}u\_\{i\}\\hat\{d\}\_\{S\_\{i\}\}\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}andu⊤​𝛅f=⟨f,∑iui​d^Si⟩ℱ𝒵u^\{\\top\}\\bm\{\\delta\}\_\{f\}=\\langle f,\\sum\_\{i\}u\_\{i\}\\hat\{d\}\_\{S\_\{i\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\. By Cauchy–Schwarz inℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\},

\(u⊤​𝜹f\)2≤‖f‖ℱ𝒵2​‖∑iui​d^Si‖ℱ𝒵2=‖f‖ℱ𝒵2​u⊤​K~​u,\(u^\{\\top\}\\bm\{\\delta\}\_\{f\}\)^\{2\}\\;\\leq\\;\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\,\\Big\\\|\\sum\_\{i\}u\_\{i\}\\hat\{d\}\_\{S\_\{i\}\}\\Big\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\;=\\;\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\,u^\{\\top\}\\widetilde\{K\}\\,u,where the last equality uses⟨d^Si,d^Sj⟩ℱ𝒵=K~i​j\\langle\\hat\{d\}\_\{S\_\{i\}\},\\hat\{d\}\_\{S\_\{j\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}=\\widetilde\{K\}\_\{ij\}\. Applying this withu=e^ju=\\hat\{e\}\_\{j\}forj=1,…,mj=1,\\ldots,mand summing,

∑j=1m\(e^j⊤​𝜹f\)2≤‖f‖ℱ𝒵2​∑j=1me^j⊤​K~​e^j=‖f‖ℱ𝒵2​tr​\(Pm​K~​Pm\)\.\\sum\_\{j=1\}^\{m\}\(\\hat\{e\}\_\{j\}^\{\\top\}\\bm\{\\delta\}\_\{f\}\)^\{2\}\\;\\leq\\;\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\,\\sum\_\{j=1\}^\{m\}\\hat\{e\}\_\{j\}^\{\\top\}\\widetilde\{K\}\\hat\{e\}\_\{j\}\\;=\\;\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\\,\\mathrm\{tr\}\\\!\\big\(P\_\{m\}\\widetilde\{K\}P\_\{m\}\\big\)\.The left\-hand side equals‖Pm​𝛅f‖22=∑i=1n\(Pm​𝛅f\)i2\\\|P\_\{m\}\\bm\{\\delta\}\_\{f\}\\\|\_\{2\}^\{2\}=\\sum\_\{i=1\}^\{n\}\(P\_\{m\}\\bm\{\\delta\}\_\{f\}\)\_\{i\}^\{2\}since\{e^j\}j=1m\\\{\\hat\{e\}\_\{j\}\\\}\_\{j=1\}^\{m\}is an orthonormal basis for the range ofPmP\_\{m\}\. Dividing bynn,

\(17\)1n​∑i=1n\(Pm​𝜹f\)i2≤‖f‖ℱ𝒵2n​tr​\(Pm​K~​Pm\)\.\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(P\_\{m\}\\bm\{\\delta\}\_\{f\}\)\_\{i\}^\{2\}\\;\\leq\\;\\frac\{\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\}\{n\}\\,\\mathrm\{tr\}\\\!\\big\(P\_\{m\}\\widetilde\{K\}P\_\{m\}\\big\)\.
Step 4: Combine\.Combining \([16](https://arxiv.org/html/2608.10470#A1.E16)\) \(which givestr​\(Pm​K~​Pm\)≤n​HSIC^/λ^m\\mathrm\{tr\}\(P\_\{m\}\\widetilde\{K\}P\_\{m\}\)\\leq n\\,\\widehat\{\\mathrm\{HSIC\}\}/\\hat\{\\lambda\}\_\{m\}\) with \([17](https://arxiv.org/html/2608.10470#A1.E17)\),

1n​∑i=1n\(Pm​𝜹f\)i2≤‖f‖ℱ𝒵2λ^m​HSIC^​\(Z,S\),\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(P\_\{m\}\\bm\{\\delta\}\_\{f\}\)\_\{i\}^\{2\}\\;\\leq\\;\\frac\{\\\|f\\\|^\{2\}\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\}\{\\hat\{\\lambda\}\_\{m\}\}\\,\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\),which is the second inequality of \([8](https://arxiv.org/html/2608.10470#S4.E8)\) sincem^f=Pm​𝛅f\\widehat\{m\}\_\{f\}=P\_\{m\}\\bm\{\\delta\}\_\{f\}\. The first inequality,Δ^GDP​\(f\)2=\(n−1​∑i\|\(m^f\)i\|\)2≤n−1​∑i\(m^f\)i2\\widehat\{\\Delta\}\_\{\\mathrm\{GDP\}\}\(f\)^\{2\}=\\big\(n^\{\-1\}\\sum\_\{i\}\|\(\\widehat\{m\}\_\{f\}\)\_\{i\}\|\\big\)^\{2\}\\leq n^\{\-1\}\\sum\_\{i\}\(\\widehat\{m\}\_\{f\}\)\_\{i\}^\{2\}, is the Cauchy–Schwarz inequality \(equivalently, Jensen applied tox↦x2x\\mapsto x^\{2\}\)\.

The population counterpart of this finite\-sample spectral control is the upper bound of Theorem[4\.8](https://arxiv.org/html/2608.10470#S4.Thmtheorem8), which handles the compact kernel operator on𝒮\\mathcal\{S\}through its spectral tailρm\\rho\_\{m\}without assuming a spectral gap\.

## Appendix BAdditional Notation and Technical Lemmas

###### Lemma B\.1\(Fubini’s theorem for RKHS\-valued integrals\)\.

Letkkbe a bounded kernel,supx∈𝒳k​\(x,x\)≤κ<∞\\sup\_\{x\\in\\mathcal\{X\}\}k\(x,x\)\\leq\\kappa<\\infty, with RKHSℱ\\mathcal\{F\}\. For any random variableXXon𝒳\\mathcal\{X\}, the mean embeddingμX=𝔼​\[k​\(X,⋅\)\]\\mu\_\{X\}=\\mathbb\{E\}\[k\(X,\\cdot\)\]exists as a Bochner integral inℱ\\mathcal\{F\}\. Moreover, for anyℱ\\mathcal\{F\}\-valued Bochner\-integrable random elementUUand anyv∈ℱv\\in\\mathcal\{F\},⟨𝔼​\[U\],v⟩ℱ=𝔼​\[⟨U,v⟩ℱ\]\\langle\\mathbb\{E\}\[U\],v\\rangle\_\{\\mathcal\{F\}\}=\\mathbb\{E\}\[\\langle U,v\\rangle\_\{\\mathcal\{F\}\}\]; in particular⟨μX,v⟩ℱ=𝔼​\[v​\(X\)\]\\langle\\mu\_\{X\},v\\rangle\_\{\\mathcal\{F\}\}=\\mathbb\{E\}\[v\(X\)\]\.

###### Proof B\.2\.

By the reproducing property,

‖k​\(x,⋅\)‖ℱ=k​\(x,x\)≤κ,so𝔼​\[‖k​\(X,⋅\)‖ℱ\]≤κ<∞,\\\|k\(x,\\cdot\)\\\|\_\{\\mathcal\{F\}\}=\\sqrt\{k\(x,x\)\}\\leq\\sqrt\{\\kappa\},\\quad\\text\{so\}\\quad\\mathbb\{E\}\[\\\|k\(X,\\cdot\)\\\|\_\{\\mathcal\{F\}\}\]\\leq\\sqrt\{\\kappa\}<\\infty,guaranteeing existence of the Bochner integral\. The exchange of expectation and inner product follows from continuity of the inner product together with Fubini’s theorem\.

### B\.1Discrete sensitive attribute: coincidence of the two formulations

WhenSStakes finitely many values, the joint and conditional\-integral formulations of representation\-level fairness reduce to the same aggregated two\-sample MMD object, which motivates studying the continuous case in the main text\.

###### Lemma B\.3\(Discrete sensitive attribute coincidence\)\.

Use the notation:ℱ𝒵\\mathcal\{F\}\_\{\\mathcal\{Z\}\}is the RKHS on𝒵\\mathcal\{Z\}with characteristic kernelk𝒵k\_\{\\mathcal\{Z\}\},μZ:=𝔼​\[k𝒵​\(Z,⋅\)\]∈ℱ𝒵\\mu\_\{Z\}:=\\mathbb\{E\}\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\]\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}is the marginal mean embedding, andμZ∣sk:=𝔼​\[k𝒵​\(Z,⋅\)∣S=sk\]∈ℱ𝒵\\mu\_\{Z\\mid s\_\{k\}\}:=\\mathbb\{E\}\[k\_\{\\mathcal\{Z\}\}\(Z,\\cdot\)\\mid S=s\_\{k\}\]\\in\\mathcal\{F\}\_\{\\mathcal\{Z\}\}is the conditional mean embedding\. Let𝒮=\{s1,…,sK\}\\mathcal\{S\}=\\\{s\_\{1\},\\dots,s\_\{K\}\\\}be finite withπk:=Pr⁡\(S=sk\)\>0\\pi\_\{k\}:=\\Pr\(S=s\_\{k\}\)\>0, letk𝒮k\_\{\\mathcal\{S\}\}be the strictly positive\-definite sensitive\-attribute kernel, writeδk:=μZ∣sk−μZ\\delta\_\{k\}:=\\mu\_\{Z\\mid s\_\{k\}\}\-\\mu\_\{Z\}, and letℐMMD2​\(Z;S\):=𝔼S​\[MMD2​\(PZ∣S,PZ\)\]\\mathcal\{I\}\_\{\\mathrm\{MMD\}^\{2\}\}\(Z;S\):=\\mathbb\{E\}\_\{S\}\[\\mathrm\{MMD\}^\{2\}\(P\_\{Z\\mid S\},P\_\{Z\}\)\]be the conditional\-integral functional with local discrepancyd=MMD2d=\\mathrm\{MMD\}^\{2\}\. Then

HSIC​\(Z,S\)=∑k,k′=1Kπk​πk′​k𝒮​\(sk,sk′\)​⟨δk,δk′⟩ℱ𝒵\.\\mathrm\{HSIC\}\(Z,S\)=\\sum\_\{k,k^\{\\prime\}=1\}^\{K\}\\pi\_\{k\}\\pi\_\{k^\{\\prime\}\}\\,k\_\{\\mathcal\{S\}\}\(s\_\{k\},s\_\{k^\{\\prime\}\}\)\\,\\langle\\delta\_\{k\},\\delta\_\{k^\{\\prime\}\}\\rangle\_\{\\mathcal\{F\}\_\{\\mathcal\{Z\}\}\}\.HSIC​\(Z,S\)\\mathrm\{HSIC\}\(Z,S\)andℐMMD2​\(Z;S\)\\mathcal\{I\}\_\{\\mathrm\{MMD\}^\{2\}\}\(Z;S\)both vanish if and only ifPZ∣S=sk=PZP\_\{Z\\mid S=s\_\{k\}\}=P\_\{Z\}for everykk; both reduce to aggregated two\-sample MMDs\.

###### Proof B\.4\.

ExpandHSIC\\mathrm\{HSIC\}viaμPZ,S−μPZ⊗PS=𝔼S​\[\(μZ∣S−μZ\)⊗k𝒮​\(S,⋅\)\]\\mu\_\{P\_\{Z,S\}\}\-\\mu\_\{P\_\{Z\}\\otimes P\_\{S\}\}=\\mathbb\{E\}\_\{S\}\[\(\\mu\_\{Z\\mid S\}\-\\mu\_\{Z\}\)\\otimes k\_\{\\mathcal\{S\}\}\(S,\\cdot\)\]and integrate the finite measure on𝒮\\mathcal\{S\}\.

### B\.2Bandwidth scaling of the spectral constant

## Appendix CRegularized empirical kHGR

This appendix records a regularized empirical bound linkingHSIC^\\widehat\{\\mathrm\{HSIC\}\}to a Tikhonov\-regularized empirical kHGR\. The unregularized empirical kHGR requires inverting empirical covariance operators with arbitrarily small positive eigenvalues, so a meaningful finite\-sample statement is naturally regularized; seeFukumizu et al\. \([2007](https://arxiv.org/html/2608.10470#bib.bib10)\)for the kCCA interpretation\. The FRHSIC guarantees in the main text do not depend on this result\.

#### Empirical covariance operators

Define the empirical centered cross\-covariance operatorΣ^Z​S:ℱ𝒮→ℱ𝒵\\hat\{\\Sigma\}\_\{ZS\}:\\mathcal\{F\}\_\{\\mathcal\{S\}\}\\to\\mathcal\{F\}\_\{\\mathcal\{Z\}\}by

Σ^Z​S=1n​∑i=1n\(k𝒵​\(Zi,⋅\)−μ^Z\)⊗\(k𝒮​\(Si,⋅\)−μ^S\),\\hat\{\\Sigma\}\_\{ZS\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\big\(k\_\{\\mathcal\{Z\}\}\(Z\_\{i\},\\cdot\)\-\\hat\{\\mu\}\_\{Z\}\\big\)\\otimes\\big\(k\_\{\\mathcal\{S\}\}\(S\_\{i\},\\cdot\)\-\\hat\{\\mu\}\_\{S\}\\big\),whereμ^Z,μ^S\\hat\{\\mu\}\_\{Z\},\\hat\{\\mu\}\_\{S\}are the empirical mean embeddings, and defineΣ^Z​Z,Σ^S​S\\hat\{\\Sigma\}\_\{ZZ\},\\hat\{\\Sigma\}\_\{SS\}analogously\. All three operators are finite\-rank \(rank at mostnn\), hence Hilbert–Schmidt without further assumptions on the kernels\. By a direct expansion,HSIC^​\(Z,S\)=‖Σ^Z​S‖HS2\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)=\\\|\\hat\{\\Sigma\}\_\{ZS\}\\\|\_\{\\mathrm\{HS\}\}^\{2\}\.

#### Regularized empirical kHGR

For ridge parametersηZ,ηS\>0\\eta\_\{Z\},\\eta\_\{S\}\>0, define the Tikhonov\-regularized empirical kHGR

kHGR^ηZ,ηS​\(Z,S\):=‖\(Σ^Z​Z\+ηZ​I\)−1/2​Σ^Z​S​\(Σ^S​S\+ηS​I\)−1/2‖op\.\\widehat\{\\mathrm\{kHGR\}\}\_\{\\eta\_\{Z\},\\eta\_\{S\}\}\(Z,S\)\\;:=\\;\\big\\\|\(\\hat\{\\Sigma\}\_\{ZZ\}\+\\eta\_\{Z\}I\)^\{\-1/2\}\\,\\hat\{\\Sigma\}\_\{ZS\}\\,\(\\hat\{\\Sigma\}\_\{SS\}\+\\eta\_\{S\}I\)^\{\-1/2\}\\big\\\|\_\{\\mathrm\{op\}\}\.
###### Proposition C\.1\(Regularized empirical sandwich bound\)\.

Letk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}be characteristic kernels and letηZ,ηS\>0\\eta\_\{Z\},\\eta\_\{S\}\>0\. Then

\(18\)kHGR^ηZ,ηS​\(Z,S\)2≤HSIC^​\(Z,S\)ηZ​ηS\.\\widehat\{\\mathrm\{kHGR\}\}\_\{\\eta\_\{Z\},\\eta\_\{S\}\}\(Z,S\)^\{2\}\\;\\leq\\;\\frac\{\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)\}\{\\eta\_\{Z\}\\,\\eta\_\{S\}\}\.

###### Proof C\.2\.

SetA=\(Σ^Z​Z\+ηZ​I\)−1/2A=\(\\hat\{\\Sigma\}\_\{ZZ\}\+\\eta\_\{Z\}I\)^\{\-1/2\},B=Σ^Z​SB=\\hat\{\\Sigma\}\_\{ZS\},C=\(Σ^S​S\+ηS​I\)−1/2C=\(\\hat\{\\Sigma\}\_\{SS\}\+\\eta\_\{S\}I\)^\{\-1/2\}\. Each operatorΣ^Z​Z,Σ^S​S\\hat\{\\Sigma\}\_\{ZZ\},\\hat\{\\Sigma\}\_\{SS\}is positive semi\-definite, soΣ^Z​Z\+ηZ​I⪰ηZ​I\\hat\{\\Sigma\}\_\{ZZ\}\+\\eta\_\{Z\}I\\succeq\\eta\_\{Z\}IandΣ^S​S\+ηS​I⪰ηS​I\\hat\{\\Sigma\}\_\{SS\}\+\\eta\_\{S\}I\\succeq\\eta\_\{S\}I\. Consequently

‖A‖op≤1/ηZ,‖C‖op≤1/ηS\.\\\|A\\\|\_\{\\mathrm\{op\}\}\\;\\leq\\;1/\\sqrt\{\\eta\_\{Z\}\},\\qquad\\\|C\\\|\_\{\\mathrm\{op\}\}\\;\\leq\\;1/\\sqrt\{\\eta\_\{S\}\}\.By the operator\-norm≤\\leqHilbert–Schmidt\-norm inequality and the standard submultiplicative bound‖A​B​C‖op≤‖A‖op​‖B‖HS​‖C‖op\\\|ABC\\\|\_\{\\mathrm\{op\}\}\\leq\\\|A\\\|\_\{\\mathrm\{op\}\}\\,\\\|B\\\|\_\{\\mathrm\{HS\}\}\\,\\\|C\\\|\_\{\\mathrm\{op\}\}\(valid because the Hilbert–Schmidt norm dominates the operator norm\),

kHGR^ηZ,ηS​\(Z,S\)=‖A​B​C‖op≤‖A‖op​‖B‖HS​‖C‖op≤‖Σ^Z​S‖HSηZ​ηS\.\\widehat\{\\mathrm\{kHGR\}\}\_\{\\eta\_\{Z\},\\eta\_\{S\}\}\(Z,S\)=\\\|ABC\\\|\_\{\\mathrm\{op\}\}\\;\\leq\\;\\\|A\\\|\_\{\\mathrm\{op\}\}\\,\\\|B\\\|\_\{\\mathrm\{HS\}\}\\,\\\|C\\\|\_\{\\mathrm\{op\}\}\\;\\leq\\;\\frac\{\\\|\\hat\{\\Sigma\}\_\{ZS\}\\\|\_\{\\mathrm\{HS\}\}\}\{\\sqrt\{\\eta\_\{Z\}\\,\\eta\_\{S\}\}\}\.Squaring and usingHSIC^​\(Z,S\)=‖Σ^Z​S‖HS2\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)=\\\|\\hat\{\\Sigma\}\_\{ZS\}\\\|\_\{\\mathrm\{HS\}\}^\{2\}gives \([18](https://arxiv.org/html/2608.10470#A3.E18)\)\.

The proposition makes no claim about the unregularized limitηZ,ηS↓0\\eta\_\{Z\},\\eta\_\{S\}\\downarrow 0, for which the spectrum of the empirical covariance operators must be controlled separately\. Population\-level zero\-equivalence of HSIC and kHGR under characteristic kernels is independent of this regularization choice\.

## Appendix DExperimental Details

### D\.1Datasets

#### Adult Income

The UCI Adult dataset containsn≈30,000n\\approx 30\{,\}000records \(after removing missing values\)\. Features include numeric variables and one\-hot encoded categoricals \(d=107d=107after encoding\)\. The target is whether income exceeds $50K/year\. The sensitive attribute is age \(continuous, range 17–90\)\. All features and the sensitive attribute are min\-max scaled to\[0,1\]\[0,1\]\.

#### ACS Income

The ACS Income dataset\(Ding et al\.,[2021](https://arxiv.org/html/2608.10470#bib.bib9)\)is drawn from the 2018 American Community Survey \(California\) via the folktables package\. We subsamplen=20,000n=20\{,\}000individuals\. The target is whether income exceeds $50K/year\. The sensitive attribute is age \(continuous, range 17–94\)\. Features include 9 demographic and employment variables\.

#### MEPS

The Medical Expenditure Panel Survey \(Panel 19, 2015\)\(Romano et al\.,[2020](https://arxiv.org/html/2608.10470#bib.bib31)\)containsn≈13,000n\\approx 13\{,\}000respondents after removing records with missing values \(negative sentinel codes\)\. The target is healthcare utilization≥10\\geq 10visits \(binary classification\)\. The sensitive attribute is age \(continuous, range 17–85\)\. Features include 19 variables covering demographics, health conditions, insurance status, and income\.

#### Communities & Crime

The UCI Communities & Crime dataset containsn≈2,000n\\approx 2\{,\}000communities\. The target is violent crimes per population \(regression\)\. The sensitive attribute is racial composition \(continuous\)\. Columns with\>20%\{\>\}20\\%missing values are dropped; remaining missing values are imputed with column medians\.

#### COMPAS

The ProPublica COMPAS dataset containsn≈6,000n\\approx 6\{,\}000defendants after standard filtering\(Angwin et al\.,[2016](https://arxiv.org/html/2608.10470#bib.bib2)\)\. The target is two\-year recidivism \(binary classification\)\. The sensitive attribute is age \(continuous\)\. Features include prior counts, charge degree \(one\-hot\), sex, and race \(one\-hot\), yieldingd=16d=16features\.

### D\.2Model Architecture

FollowingKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\), the encoderhhis a 2\-layer MLP:dX→50→50d\_\{X\}\\to 50\\to 50with SELU activations \(hidden dimensionH=50H=50, output representation dimensionZ=50Z=50\)\. The prediction headffis a linear layer:50→150\\to 1\(no hidden layers\)\. This architecture matches FREM exactly, ensuring a fair comparison\. The adversary in LAFTR and ADV uses the same 2\-layer structure \(50→50→nout50\\to 50\\to n\_\{\\mathrm\{out\}\}\) with SELU activation\.

### D\.3Training Details

All models are trained for200200epochs with the Adam optimizer \(learning rate10−310^\{\-3\}, defaultβ1=0\.9\\beta\_\{1\}=0\.9,β2=0\.999\\beta\_\{2\}=0\.999,ε=10−8\\varepsilon=10^\{\-8\}, weight decay0\), batch size256256, and the standard PyTorch default initialization\. For FRHSIC, kernel bandwidths are set via the median heuristic:σ𝒮\\sigma\_\{\\mathcal\{S\}\}is computed once on the training data, andσ𝒵\\sigma\_\{\\mathcal\{Z\}\}is recomputed every 20 epochs on the current mini\-batch of encoded representations\. For baselines \(FREM, Reg\-GDP, etc\.\), we use fixedσZ=1\.0\\sigma\_\{Z\}=1\.0for theZZ\-kernel after min\-max scaling to\[0,1\]\[0,1\], followingKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\); the additionalSS\-smoothing bandwidths used by FREM \(γ=0\.5\\gamma=0\.5\) and Reg\-GDP \(Nadaraya–Watson bandwidth0\.20\.2\) are taken from their source publications and described per\-method below\. We do not modify per\-method bandwidth conventions, ensuring each method is evaluated as published\.

#### Train/validation/test splits

For all real datasets we use an80%/20%80\\%/20\\%random train/test split\. Hyperparameter selection \(i\.e\., choosingλ\\lambdaalong the Pareto frontier\) is performed using the same training split as the model fit; for each method and each dataset we sweepλ\\lambdaover the full grid and report results across the resulting Pareto frontier \(no separate validation split is used to pick a singleλ\\lambda, since the goal of the comparison is to characterize the fairness–accuracy tradeoff curve, not to choose one operating point\)\. We repeat 5 times with different random splits and report mean±\\pmstandard deviation on the held\-out test set\.

#### Regularization\-strength \(λ\\lambda\) grids

For all methods on all real datasets, we sweep

λ∈\{0\.1,1\.0,10\.0,100\.0,500\.0\}\.\\lambda\\in\\\{0\.1,\\;1\.0,\\;10\.0,\\;100\.0,\\;500\.0\\\}\.For the Equal Opportunity and multi\-sensitive\-attribute experiments \(Appendix[E](https://arxiv.org/html/2608.10470#A5)\) the grid isλ∈\{0\.1,1\.0,10\.0,50\.0\}\\lambda\\in\\\{0\.1,1\.0,10\.0,50\.0\\\}\.

### D\.4Baseline Implementations and Hyperparameter Grids

#### Reg\-GDP

We implement the GDP regularizer fromJiang et al\. \([2022](https://arxiv.org/html/2608.10470#bib.bib15)\)using kernel\-smoothed conditional expectations on a grid of 30 evenly\-spaced points over the range ofSS\. Reg\-GDP uses two distinct bandwidths: a GaussianZZ\-kernel bandwidthσZ=1\.0\\sigma\_\{Z\}=1\.0\(matching the FREM convention ofKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\)on min\-max\-scaled features\) and a Nadaraya–Watson smoothing bandwidth0\.20\.2onSSfor the conditional expectation; both follow the values reported byJiang et al\. \([2022](https://arxiv.org/html/2608.10470#bib.bib15)\)\.

#### FREM

We implement the weighted EIPM estimator fromKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\)using the MMD discriminator\. Kernel\-smoothed weights are computed with a Gaussian kernel onSSwith bandwidthγ=0\.5\\gamma=0\.5\(followingKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\)\)\. For computational tractability we subsamplenanchor=32n\_\{\\mathrm\{anchor\}\}=32anchor points per batch for the EIPM computation; see Appendix[D\.5](https://arxiv.org/html/2608.10470#A4.SS5)below for details\.

#### LAFTR

We bin the continuous sensitive attribute into 4 quartiles and apply the adversarial FRL method ofMadras et al\. \([2018](https://arxiv.org/html/2608.10470#bib.bib24)\)\. The adversary is trained with alternating gradient steps \(one adversary step per encoder step\)\. The adversary is a 2\-layer MLP \(50→50→450\\to 50\\to 4\) with SELU activations\.

#### ADV

The continuous\-target adversary ofGrari et al\. \([2022](https://arxiv.org/html/2608.10470#bib.bib11)\), with the same 2\-layer architecture as LAFTR but with a single scalar output \(predictingSS\); trained with one adversary step per encoder step\.

#### MMD\-binned

Bins continuousSSintonbins=10n\_\{\\mathrm\{bins\}\}=10equal\-width bins and applies pairwise MMD across bins; bin centers serve as the smoothing grid\.

#### dCor

We use the unbiased empirical distance correlation ofSzékely et al\. \([2007](https://arxiv.org/html/2608.10470#bib.bib36)\)as the fairness penalty; no further hyperparameters beyondλ\\lambda\.

### D\.5FREM 32\-anchor subsampling

The FREM EIPM objective ofKong et al\. \([2025](https://arxiv.org/html/2608.10470#bib.bib18)\)computes, at each mini\-batch, an integral overssof a weighted MMD term:

EIPM^n​\(Z;S\)=∫𝒮MMD^2​\(P^Z∣S=s,P^Z\)​𝑑P^S​\(s\)\.\\widehat\{\\mathrm\{EIPM\}\}\_\{n\}\(Z;S\)=\\int\_\{\\mathcal\{S\}\}\\widehat\{\\mathrm\{MMD\}\}^\{2\}\\big\(\\widehat\{P\}\_\{Z\\mid S=s\},\\,\\widehat\{P\}\_\{Z\}\\big\)\\,d\\widehat\{P\}\_\{S\}\(s\)\.Estimating this expectation by Monte Carlo with allnntraining points as anchors would requireO​\(n3\)O\(n^\{3\}\)work per batch \(ann×nn\\times nkernel matrix evaluated at each ofnnanchors\)\. For computational tractability we instead drawnanchor=32n\_\{\\mathrm\{anchor\}\}=32uniformly\-random anchor points per mini\-batch and average the MMD terms over those anchors\. With batch size256256this is a32/256=12\.5%32/256=12\.5\\%Monte Carlo subsample and reduces the per\-batch cost toO​\(nanchor⋅n2\)O\(n\_\{\\mathrm\{anchor\}\}\\cdot n^\{2\}\)\. The number3232was chosen to match the FREM implementation in our codebase \(seeexperiments/real\_data\.py, functiontrain\_frem\); we report results withnanchor=32n\_\{\\mathrm\{anchor\}\}=32throughout\.

#### FREM anchor\-subsampling sensitivity \(limitation\)

The 32\-anchor subsampling we use for FREM is a computational approximation; an anchor\-count sweep is the natural next step to characterize the sensitivity of FREM’s reported Pareto frontier to the anchor budget, and we plan to include such a sweep in a follow\-up version of this work\. The per\-epoch wall\-clock cost grows linearly with the anchor count, so the principal cost of the sweep is compute rather than implementation\.

#### Baseline bandwidth sensitivity \(limitation\)

The bandwidths reported in Appendix[D](https://arxiv.org/html/2608.10470#A4)\(FRHSIC median heuristic, FREMγ=0\.5\\gamma=0\.5andσZ=1\.0\\sigma\_\{Z\}=1\.0, Reg\-GDP Nadaraya–Watson bandwidth0\.20\.2andσZ=1\.0\\sigma\_\{Z\}=1\.0\) are the values reported in the source publications\. We retain published bandwidths in this paper to avoid favoring any particular method by retuning, but a full bandwidth\-sensitivity sweep for the FREM and Reg\-GDP baselines is the natural next step, and we plan to include such a sweep in a follow\-up version of this work\.

#### Paired significance test \(limitation\)

We report mean and standard deviation acrossn=5n=5random splits in Table[1](https://arxiv.org/html/2608.10470#S6.T1)but do not run a formal paired test \(e\.g\., Wilcoxon signed\-rank\), since55paired observations is too few for reliable inference at conventional significance levels\. Increasing the number of repeats to enable a credible paired comparison is the natural next step and we plan to include such an analysis in a follow\-up version of this work\.

### D\.6Reproducibility checklist

We summarize the elements needed to reproduce the experiments end to end\.

- •Random seed\.A single base seed`SEED = 42`is set at the top of each experiment script for`numpy`,`torch`, and the train/test split RNG\. The 5 repeats use seeds𝚂𝙴𝙴𝙳,𝚂𝙴𝙴𝙳\+1,…,𝚂𝙴𝙴𝙳\+4\\verb\|SEED\|,\\verb\|SEED\|\+1,\\ldots,\\verb\|SEED\|\+4\. See`experiments/real\_data\.py:27`and`experiments/utils\.py`\.
- •Train/validation/test split\.80%/20%80\\%/20\\%random train/test split; no separate validation split \(rationale above\)\. Repeated 5 times with different splits\.
- •Preprocessing\.Min–max scaling for the features and the sensitive attribute is fit on the*training split only*and then applied to the held\-out test split \(`experiments/real\_data\.py`,`run\_single\_dataset`\)\. All cells of Table[1](https://arxiv.org/html/2608.10470#S6.T1)in the main text are reported under this leakage\-free scaling protocol; the resulting numbers agree with our earlier leakage\-affected numbers to within standard deviations on essentially every cell, with the most visible shifts in the MMD and ADV rows of Crime \(where the small sample sizen≈2000n\\approx 2000amplifies any scaling difference\)\. Categoricals are one\-hot encoded; rows with missing values are dropped except in Communities & Crime, where columns with\>20%\{\>\}20\\%missing are dropped and remaining missing values are imputed with column medians\.
- •Model architecture\.Encoderhh: 2\-layer MLPdX→50→50d\_\{X\}\\to 50\\to 50with SELU activations\. Predictorff: linear50→150\\to 1\. Adversaries \(LAFTR / ADV / MMD\-binned\): 2\-layer MLP50→50→nout50\\to 50\\to n\_\{\\mathrm\{out\}\}with SELU\.
- •Optimizer\.Adam, learning rate10−310^\{\-3\},β1=0\.9,β2=0\.999\\beta\_\{1\}=0\.9,\\beta\_\{2\}=0\.999,ε=10−8\\varepsilon=10^\{\-8\}, weight decay0\. PyTorch defaults\.
- •Batch size and epochs\.Batch size256256,200200epochs\.
- •Kernel bandwidths\.FRHSIC: median heuristic forσ𝒮\\sigma\_\{\\mathcal\{S\}\}\(fixed at start of training, on training data\) and forσ𝒵\\sigma\_\{\\mathcal\{Z\}\}\(recomputed every 20 epochs on the current mini\-batch\)\. Baselines: fixedσZ=1\.0\\sigma\_\{Z\}=1\.0on min\-max\-scaledZZ, FREMγ=0\.5\\gamma=0\.5, Reg\-GDP NW bandwidth0\.20\.2\.
- •λ\\lambdagrids\.Main experiments:λ∈\{0\.1,1\.0,10\.0,100\.0,500\.0\}\\lambda\\in\\\{0\.1,1\.0,10\.0,100\.0,500\.0\\\}\. EO and multi\-sensitive:λ∈\{0\.1,1\.0,10\.0,50\.0\}\\lambda\\in\\\{0\.1,1\.0,10\.0,50\.0\\\}\.
- •Baseline hyperparameter grids\.See Appendix[D](https://arxiv.org/html/2608.10470#A4), “Baseline Implementations”; all baselines use the same encoder, predictor, optimizer, batch size, and number of epochs as FRHSIC, varying only the per\-method fairness\-penalty hyperparameters listed above\. The adversary step ratio \(LAFTR, ADV\) is fixed at1:11\\\!:\\\!1\.
- •FREM anchor count\.nanchor=32n\_\{\\mathrm\{anchor\}\}=32uniformly\-sampled anchors per mini\-batch; see Appendix[D\.5](https://arxiv.org/html/2608.10470#A4.SS5)\.
- •Hardware\.All experiments were run on a single workstation with one NVIDIA RTX\-class GPU; the experiment code falls back to CPU if no CUDA device is available \(see the device selector in`experiments/utils\.py`\)\. Runtime numbers in Section[6](https://arxiv.org/html/2608.10470#S6)are wall\-clock per\-epoch on this single device, averaged over 5 epochs after a warm\-up epoch\. Training the fullλ\\lambda\-sweep across all methods and datasets fits within∼\\sim24 GPU\-hours\.
- •Software\.Python 3\.11, PyTorch 2\.x, NumPy 1\.26, scikit\-learn 1\.3, folktables \(for ACS Income\)\. See`experiments/README\.md`for the exact versions used at submission time\.
- •Code availability\.All scripts to reproduce the figures and tables are in the`experiments/`directory of the public code repository \([https://github\.com/Yijin911/FRHSIC](https://github.com/Yijin911/FRHSIC)\); each main\-paper figure/table is generated by a single named script \(e\.g\.`run\_single\_method\.py`,`regen\_pareto\_mi\.py`,`runtime\.py`,`convergence\.py`\)\.

### D\.7Pareto summary: lowest GDP within 1% of the unfair\-baseline performance

For a single\-number summary of the fairness–accuracy tradeoff that complements the matched\-performance comparison in Section[6](https://arxiv.org/html/2608.10470#S6), Table[3](https://arxiv.org/html/2608.10470#A4.T3)reports, for each \(method, dataset\) pair, the operating point on theλ\\lambda\-sweep that attains the*lowest GDP*subject to the constraint that the prediction performance is within1%1\\%of the unfair baseline \(Acc≥0\.99⋅AccUnfair\\geq 0\.99\\cdot\\mathrm\{Acc\}\_\{\\mathrm\{Unfair\}\}for classification; MSE≤1\.01⋅MSEUnfair\\leq 1\.01\\cdot\\mathrm\{MSE\}\_\{\\mathrm\{Unfair\}\}for regression\)\. Entries are read off the Pareto sweep underlying Figure[3](https://arxiv.org/html/2608.10470#S6.F3)\. “—” denotes that noλ\\lambdain our grid satisfies the constraint for that method on that dataset\. Values are means across55random splits\.

Table 3:Pareto summary:\(Acc / MSE,GDP\)\(\\text\{Acc / MSE\},\\text\{GDP\}\)at the lowest\-GDP operating point with prediction performance within1%1\\%of the unfair baseline\. Adult/ACS/MEPS/COMPAS report Acc; Crime reportsMSE×102\\mathrm\{MSE\}\\times 10^\{2\}\. “—” indicates noλ\\lambdain the sweep satisfies the constraint\. Read alongside Figure[3](https://arxiv.org/html/2608.10470#S6.F3), since a tight1%1\\%band can exclude operating points that are visually close on the frontier\.MethodAdultACS IncomeMEPSCrimeCOMPASUnfair \(ref\.\)\(\.849\.849,\.749\.749\)\(\.790\.790,\.514\.514\)\(\.804\.804,\.626\.626\)\(1\.911\.91,\.029\.029\)\(\.654\.654,\.123\.123\)FRHSIC \(Ours\)\(\.845\.845,\.232\.232\)\(\.792\.792,\.399\.399\)\(\.803\.803,\.517\.517\)\(1\.911\.91,\.029\.029\)\(\.655\.655,\.107\.107\)FREM\(\.845\.845,\.167\.167\)\(\.787\.787,\.274\.274\)\(\.800\.800,\.317\.317\)\(1\.871\.87,\.023\.023\)\(\.648\.648,\.101\.101\)Reg\-GDP\(\.842\.842,\.082\.082\)\(\.787\.787,\.292\.292\)\(\.801\.801,\.370\.370\)\(1\.911\.91,\.025\.025\)\(\.650\.650,\.079\.079\)ADV\(\.841\.841,\.466\.466\)\(\.785\.785,\.458\.458\)\(\.803\.803,\.480\.480\)\(1\.921\.92,\.029\.029\)\(\.649\.649,\.095\.095\)MMD \(binned\)\(\.847\.847,\.126\.126\)\(\.783\.783,\.157\.157\)\(\.800\.800,\.407\.407\)—\(\.651\.651,\.082\.082\)LAFTR \(binned\)\(\.842\.842,\.385\.385\)\(\.787\.787,\.474\.474\)\(\.803\.803,\.530\.530\)\(1\.891\.89,\.029\.029\)\(\.655\.655,\.122\.122\)dCor\(\.846\.846,\.194\.194\)\(\.783\.783,\.203\.203\)\(\.800\.800,\.487\.487\)\(1\.921\.92,\.023\.023\)\(\.653\.653,\.091\.091\)#### Caveats

The 1% accuracy band is intentionally tight, and on some datasets \(notably COMPAS, where multiple methods cluster near the unfair\-baseline accuracy\) the band excludes operating points that are visually close on the Pareto frontier \(Figure[3](https://arxiv.org/html/2608.10470#S6.F3)\)\. Reg\-GDP can drive GDP to near zero on Adult and ACS, but only at accuracy losses well outside the 1% band; those points are visible on Figure[3](https://arxiv.org/html/2608.10470#S6.F3)but do not appear in the table above\. The single\-number summary should therefore be read alongside the full Pareto curves rather than as a standalone ranking\.

## Appendix EAdditional Experimental Results

### E\.1Synthetic validation of the conditional\-gap bound

On a controlled synthetic classification setup, enforcing HSIC drives GDP monotonically to zero at stable accuracy, confirming the conditional\-gap bound of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\. We generaten=5000n=5000samples withS∼Uniform​\(0,1\)S\\sim\\mathrm\{Uniform\}\(0,1\),X=\(S\+ϵ1,ϵ2\)X=\(S\+\\epsilon\_\{1\},\\epsilon\_\{2\}\)whereϵ1∼𝒩​\(0,0\.09\)\\epsilon\_\{1\}\\sim\\mathcal\{N\}\(0,0\.09\),ϵ2∼𝒩​\(0,1\)\\epsilon\_\{2\}\\sim\\mathcal\{N\}\(0,1\), andY∼Bernoulli​\(1/\(1\+e−X1\)\)Y\\sim\\mathrm\{Bernoulli\}\(1/\(1\+e^\{\-X\_\{1\}\}\)\)\. The encoderhhis a 2\-layer MLP \(hidden dimension 64, representation dimension 8\) and the prediction headffis linear; a smaller architecture than the real\-data experiments isolates the effect of the HSIC penalty in a controlled setting\. We train with objectiveℒCE​\(f∘h\)\+λ⋅HSIC^​\(Z,S\)\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(f\\circ h\)\+\\lambda\\cdot\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)forλ∈\{0,0\.1,0\.5,1,5,10,50\}\\lambda\\in\\\{0,0\.1,0\.5,1,5,10,50\\\}\. Asλ\\lambdaincreases, GDP decreases monotonically toward zero \(Figure[6](https://arxiv.org/html/2608.10470#A5.F6)a\), GDP and HSIC decrease together in a monotone relationship consistent with the upper bound in Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\(Figure[6](https://arxiv.org/html/2608.10470#A5.F6)c\), and accuracy remains stable \(around0\.620\.62\), indicating that the sensitive attribute contributes primarily to unfair discrimination rather than useful predictive signal in this setting\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x6.png)Figure 6:Synthetic validation\. \(a\) GDP decreases asλ\\lambdaincreases\. \(b\) Accuracy remains stable across regularization strengths\. \(c\) GDP vs\. HSIC shows a monotone relationship, consistent with the conditional\-gap\-control interpretation of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\(the plot displays monotonic dependence, not the LHS/RHS of the bound\)\.
### E\.2Equal Opportunity and multiple sensitive attributes

The HSIC penalty extends to Equal Opportunity\(Madras et al\.,[2018](https://arxiv.org/html/2608.10470#bib.bib24)\), which requires fairness conditional on the positive outcomeY=1Y=1, by restricting the estimator to then1n\_\{1\}samples withYi=1Y\_\{i\}=1:

HSIC^EO​\(Z,S\)=1n12​tr​\(K1​H1​L1​H1\),\\widehat\{\\mathrm\{HSIC\}\}\_\{\\mathrm\{EO\}\}\(Z,S\)=\\frac\{1\}\{n\_\{1\}^\{2\}\}\\,\\mathrm\{tr\}\(K\_\{1\}H\_\{1\}L\_\{1\}H\_\{1\}\),whereK1,L1,H1K\_\{1\},L\_\{1\},H\_\{1\}are the kernel and centering matrices restricted to that subset\. It extends to multiple continuous sensitive attributes by taking a product kernel on𝒮1×𝒮2\\mathcal\{S\}\_\{1\}\\times\\mathcal\{S\}\_\{2\}\(Appendix[E\.3](https://arxiv.org/html/2608.10470#A5.SS3)\)\.

We evaluate the EO extension on Adult and COMPAS, comparing FRHSIC\-DP \(HSIC on all samples\) to FRHSIC\-EO \(HSIC onY=1Y=1subset only\)\. Table[4](https://arxiv.org/html/2608.10470#A5.T4)reports accuracy and EO\-GDP \(GDP restricted to theY=1Y=1subset\) acrossλ\\lambdavalues\. On COMPAS, the DP variant achieves comparable or better EO\-GDP than the EO variant at similar accuracy across allλ\\lambdavalues\. On Adult, the EO variant achieves slightly lower EO\-GDP at highλ\\lambda\(0\.352 vs\. 0\.418 atλ=50\\lambda=50\), but the DP variant simultaneously reduces full GDP from 0\.889 to 0\.240—a broader fairness guarantee\. Overall, enforcing full demographic parity via HSIC provides a strong baseline for Equal Opportunity, with the dedicated EO variant offering marginal gains on larger datasets\.

Table 4:Equal Opportunity results: FRHSIC\-DP vs\. FRHSIC\-EO\. EO\-GDP is GDP restricted toY=1Y=1samples\. FRHSIC\-DP achieves comparable EO fairness while also enforcing DP\.Datasetλ\\lambdaMethodAccEO\-GDP \(↓\\downarrow\)Adult0\.1DP0\.8510\.453EO0\.8490\.4061\.0DP0\.8460\.403EO0\.8500\.41610\.0DP0\.8460\.437EO0\.8500\.39250\.0DP0\.8400\.418EO0\.8490\.352COMPAS0\.1DP0\.6590\.067EO0\.6620\.0701\.0DP0\.6620\.070EO0\.6620\.07510\.0DP0\.6570\.064EO0\.6550\.06550\.0DP0\.5990\.033EO0\.6590\.076
### E\.3Multiple Sensitive Attributes

We test FRHSIC with two continuous sensitive attributes \(age and hours\-per\-week\) on Adult\. Table[5](https://arxiv.org/html/2608.10470#A5.T5)compares two approaches: \(1\) a product kernel on joint\(S1,S2\)\(S\_\{1\},S\_\{2\}\)\(FRHSIC\-Joint\), and \(2\) a sum of per\-attribute HSIC terms \(FRHSIC\-Sum\)\. Both approaches reduce GDP with respect to each attribute asλ\\lambdaincreases while preserving accuracy, confirming that HSIC naturally extends to the multi\-attribute setting without modification\. The Joint approach achieves slightly lower GDP atλ=50\\lambda=50\(Age:0\.3060\.306vs\.0\.6780\.678; Hours:0\.5170\.517vs\.0\.7590\.759\), likely because the product kernel captures cross\-attribute structure\. Figure[7](https://arxiv.org/html/2608.10470#A5.F7)visualizes the Pareto frontiers\.

Table 5:Multiple sensitive attributes on Adult \(age \+ hours\-per\-week\)\. GDP is reported separately for each attribute\. Both product\-kernel \(Joint\) and sum\-of\-HSIC \(Sum\) approaches reduce GDP for both attributes asλ\\lambdaincreases\.Method \(λ\\lambda\)AccGDP\(Age\) \(↓\\downarrow\)GDP\(Hours\) \(↓\\downarrow\)Unfair0\.8430\.8371\.142FRHSIC\-Joint \(λ=1\\lambda=1\)0\.8440\.7791\.029FRHSIC\-Joint \(λ=10\\lambda=10\)0\.8460\.5530\.857FRHSIC\-Joint \(λ=50\\lambda=50\)0\.8510\.3060\.517FRHSIC\-Sum \(λ=1\\lambda=1\)0\.8470\.7531\.055FRHSIC\-Sum \(λ=10\\lambda=10\)0\.8480\.4850\.826FRHSIC\-Sum \(λ=50\\lambda=50\)0\.8490\.6780\.759![Refer to caption](https://arxiv.org/html/2608.10470v1/x7.png)Figure 7:Multiple sensitive attributes: fairness–accuracy Pareto frontiers for FRHSIC\-Joint \(product kernel\) and FRHSIC\-Sum \(sum of per\-attribute HSIC\), evaluated on Adult with age and hours\-per\-week as sensitive attributes\.

## Appendix FStructural Causal Models

### F\.1Structural Causal Models and Counterfactual Fairness

FollowingPearl \([2009](https://arxiv.org/html/2608.10470#bib.bib28)\)andKusner et al\. \([2017](https://arxiv.org/html/2608.10470#bib.bib19)\), a Structural Causal Model \(SCM\) over\(X,S,Y\)\(X,S,Y\)specifies structural equations

S=US,X=fX​\(S,UX\),Y=fY​\(X,S,UY\),S=U\_\{S\},\\qquad X=f\_\{X\}\(S,U\_\{X\}\),\\qquad Y=f\_\{Y\}\(X,S,U\_\{Y\}\),with jointly independent exogenous noiseUX,US,UYU\_\{X\},U\_\{S\},U\_\{Y\}\. The counterfactual predictionY^S←s′\\widehat\{Y\}\_\{S\\leftarrow s^\{\\prime\}\}is the value ofY^\\widehat\{Y\}obtained by replacingS=USS=U\_\{S\}withS=s′S=s^\{\\prime\}in the structural equations and propagating downstream\. A predictorY^\\widehat\{Y\}is*counterfactually fair*\(Kusner et al\.,[2017](https://arxiv.org/html/2608.10470#bib.bib19)\)if, for every observable\(X=x,S=s\)\(X=x,S=s\)and every alternatives′s^\{\\prime\},

Y^S←s′=Y^S←sa\.s\. given​X=x,S=s\.\\widehat\{Y\}\_\{S\\leftarrow s^\{\\prime\}\}=\\widehat\{Y\}\_\{S\\leftarrow s\}\\quad\\text\{a\.s\.\\ given \}X=x,S=s\.

## Appendix GAdditional Theoretical Augmentations

This section collects two augmentations referenced from the main text: a necessity\-for\-counterfactual\-fairness result and a uniform\-concentration bound that bridges training\-sample HSIC to its population counterpart\.

### G\.1Necessity for Counterfactual Fairness

The joint\-distribution route to independence also clarifies the relationship between HSIC\-fairness and the causal\-fairness literature\.

###### Proposition G\.1\(HSIC\-fairness as a necessary condition for counterfactual fairness\)\.

Assume the SCM in Appendix[F\.1](https://arxiv.org/html/2608.10470#A6.SS1)\. Suppose the encoderhhis a deterministic function so thatZ=h​\(X\)=h​\(fX​\(S,UX\)\)Z=h\(X\)=h\(f\_\{X\}\(S,U\_\{X\}\)\)\. If the predictionY^=f​\(Z\)\\widehat\{Y\}=f\(Z\)is counterfactually fair \(Appendix[F\.1](https://arxiv.org/html/2608.10470#A6.SS1)\) for everyffin a classℱ\\mathcal\{F\}rich enough to identify equality of distributions in𝒵\\mathcal\{Z\}\(e\.g\.,ℱ\\mathcal\{F\}is the unit ball of an RKHS with characteristic kernelk𝒵k\_\{\\mathcal\{Z\}\}\), thenHSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0\.

###### Proof G\.2\.

Counterfactual fairness for everyf∈ℱf\\in\\mathcal\{F\}implies, in particular, that for alls′∈𝒮s^\{\\prime\}\\in\\mathcal\{S\}in the support ofSSand allf∈ℱf\\in\\mathcal\{F\},

𝔼​\[f​\(ZS←s′\)\]=𝔼​\[f​\(Z\)\],\\mathbb\{E\}\[f\(Z\_\{S\\leftarrow s^\{\\prime\}\}\)\]=\\mathbb\{E\}\[f\(Z\)\],whereZS←s′=h​\(fX​\(s′,UX\)\)Z\_\{S\\leftarrow s^\{\\prime\}\}=h\(f\_\{X\}\(s^\{\\prime\},U\_\{X\}\)\)\. SinceS=USS=U\_\{S\}andUXU\_\{X\}are independent under the SCM, and the marginal ofZS←s′Z\_\{S\\leftarrow s^\{\\prime\}\}overUXU\_\{X\}has the same distribution as the marginal ofZZgivenS=s′S=s^\{\\prime\}\(because intervening withS=s′S=s^\{\\prime\}matches the conditional givenS=s′S=s^\{\\prime\}whenSSis exogenous\(Pearl,[2009](https://arxiv.org/html/2608.10470#bib.bib28), Section 3\.2\.2\)\), we obtain

𝔼​\[f​\(Z\)∣S=s′\]=𝔼​\[f​\(Z\)\]forPS\-a\.e\.​s′\.\\mathbb\{E\}\[f\(Z\)\\mid S=s^\{\\prime\}\]=\\mathbb\{E\}\[f\(Z\)\]\\quad\\text\{for $P\_\{S\}$\-a\.e\.\\ \}s^\{\\prime\}\.By Proposition[3\.1](https://arxiv.org/html/2608.10470#S3.Thmtheorem1), this is condition \(4\), equivalent toZ⟂SZ\\perp Sand hence, for the characteristic product kernel, toHSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0\.

In words, if a representation can support counterfactually fair predictions for every \(sufficiently rich\) prediction head, the representation must already satisfy the observational\-independence condition that HSIC enforces\. The converse does not hold:HSIC​\(Z,S\)=0\\mathrm\{HSIC\}\(Z,S\)=0ensures that the marginal distribution off​\(Z\)f\(Z\)does not depend onSS, but counterfactual fairness is a strictly pointwise statement about individual\-level interventions, which can fail when the structural dependence onSSis observationally invisible \(e\.g\.,Z=S⊕UXZ=S\\oplus U\_\{X\}with binaryS,UXS,U\_\{X\}uniform yieldsZ⟂SZ\\perp Smarginally butZS←s′≠ZS←sZ\_\{S\\leftarrow s^\{\\prime\}\}\\neq Z\_\{S\\leftarrow s\}pointwise\)\. HSIC\-fairness should therefore be understood as the strongest*observational*fairness consequence of counterfactual fairness, not as a substitute for it\.

### G\.2Uniform Concentration over the Encoder Class

The empirical Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)controls the empirical conditional\-mean gap1n​∑i\(Pm​𝜹f\)i2\\frac\{1\}\{n\}\\sum\_\{i\}\(P\_\{m\}\\bm\{\\delta\}\_\{f\}\)\_\{i\}^\{2\}throughHSIC^​\(h​\(X\),S\)\\widehat\{\\mathrm\{HSIC\}\}\(h\(X\),S\)for a single encoderhh\. In practice,hhis learned from the same data, so the bound must hold uniformly over a hypothesis classℋ\\mathcal\{H\}of encoders\. We invoke Corollary 23 ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)\(an HSIC specialization of their Theorem 12\), which we adapt to the one\-sided encoder class arising in FRHSIC\. We do not reprove that result; this appendix only maps FRHSIC onto it\.

#### Mapping FRHSIC to the prior uniform concentration result

First,HSIC​\(Z,S\)\\mathrm\{HSIC\}\(Z,S\)is the squared MMD betweenPZ,SP\_\{Z,S\}andPZ⊗PSP\_\{Z\}\\otimes P\_\{S\}under the fixed product kernelk​\(\(z,s\),\(z′,s′\)\)=k𝒵​\(z,z′\)​k𝒮​\(s,s′\)k\\big\(\(z,s\),\(z^\{\\prime\},s^\{\\prime\}\)\\big\)=k\_\{\\mathcal\{Z\}\}\(z,z^\{\\prime\}\)\\,k\_\{\\mathcal\{S\}\}\(s,s^\{\\prime\}\), andHSIC^n​\(h​\(X\),S\)=n−2​tr​\(Kh​H​L​H\)\\widehat\{\\mathrm\{HSIC\}\}\_\{n\}\(h\(X\),S\)=n^\{\-2\}\\,\\mathrm\{tr\}\(K\_\{h\}HLH\)is exactly the empirical kernel\-based two\-sample statistic to which Theorem 12 ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)applies, with\(Z′,S′\)∼PZ⊗PS\(Z^\{\\prime\},S^\{\\prime\}\)\\sim P\_\{Z\}\\otimes P\_\{S\}\. Second, the encoder classℋ\\mathcal\{H\}acts only on the𝒵\\mathcal\{Z\}\-coordinate, inducing the transformed paired samples\{\(h​\(Xi\),Si\)\}\\\{\(h\(X\_\{i\}\),S\_\{i\}\)\\\}and the encoder\-indexed product\-kernel class𝒦ℋ\\mathcal\{K\}\_\{\\mathcal\{H\}\}\. Third, the boundedness and Lipschitz hypotheses needed byNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)\(Assumption 20\) are the bounded\-kernel and Lipschitz\-feature\-map conditions stated in Proposition[G\.3](https://arxiv.org/html/2608.10470#A7.Thmtheorem3), whichk𝒵k\_\{\\mathcal\{Z\}\},k𝒮k\_\{\\mathcal\{S\}\}, andh∈ℋh\\in\\mathcal\{H\}satisfy under the standing assumptions\. Fourth, bounded linear maps over a compact parameter set, and fixed\-architecture MLPs with bounded weights, bounded inputs, and Lipschitz activations, have empirical Gaussian \(equivalently, up to logarithmic factors, Rademacher\) complexity of ordern−1/2n^\{\-1/2\}, so the complexity term in the imported bound is controlled; growing\-width or unregularized networks need not satisfy this\. Data\-dependent kernel selection, which would enlarge𝒦ℋ\\mathcal\{K\}\_\{\\mathcal\{H\}\}to a composite\-kernel class, is outside the scope of this statement\.

###### Proposition G\.3\(Uniform concentration ofHSIC^\\widehat\{\\mathrm\{HSIC\}\}over an encoder class\)\.

Letℋ\\mathcal\{H\}be a class of measurable encodersh:𝒳→𝒵h:\\mathcal\{X\}\\to\\mathcal\{Z\}\. Suppose the kernelsk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}satisfy Assumption 20 ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\): they are bounded withsupzk𝒵​\(z,z\)≤ν𝒵\\sup\_\{z\}k\_\{\\mathcal\{Z\}\}\(z,z\)\\leq\\nu\_\{\\mathcal\{Z\}\}andsupsk𝒮​\(s,s\)≤ν𝒮\\sup\_\{s\}k\_\{\\mathcal\{S\}\}\(s,s\)\\leq\\nu\_\{\\mathcal\{S\}\}, and the feature mapsz↦k𝒵​\(z,⋅\)z\\mapsto k\_\{\\mathcal\{Z\}\}\(z,\\cdot\)ands↦k𝒮​\(s,⋅\)s\\mapsto k\_\{\\mathcal\{S\}\}\(s,\\cdot\)are Lipschitz with constantsℓ𝒵,ℓ𝒮\>0\\ell\_\{\\mathcal\{Z\}\},\\ell\_\{\\mathcal\{S\}\}\>0, respectively \(the constantsν𝒵,ν𝒮,ℓ𝒵,ℓ𝒮\\nu\_\{\\mathcal\{Z\}\},\\nu\_\{\\mathcal\{S\}\},\\ell\_\{\\mathcal\{Z\}\},\\ell\_\{\\mathcal\{S\}\}are exactly those of Theorem[5\.1](https://arxiv.org/html/2608.10470#S5.Thmtheorem1)\)\. Let

𝒢^n​\(ℋ\):=𝔼ξ​\[suph∈ℋ1n​∑i=1n⟨ξi,h​\(Xi\)⟩\|\{Xi\}i=1n\]\\hat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\):=\\mathbb\{E\}\_\{\\xi\}\\\!\\left\[\\sup\_\{h\\in\\mathcal\{H\}\}\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\langle\\xi\_\{i\},h\(X\_\{i\}\)\\rangle\\,\\bigg\|\\,\\\{X\_\{i\}\\\}\_\{i=1\}^\{n\}\\right\]denote the empirical Gaussian complexity ofℋ\\mathcal\{H\}\(Ni and Huo,[2024](https://arxiv.org/html/2608.10470#bib.bib27), Equation 3\.5\), whereξ1,…,ξn\\xi\_\{1\},\\ldots,\\xi\_\{n\}are i\.i\.d\.𝒩​\(0,Idim\(𝒵\)\)\\mathcal\{N\}\(0,I\_\{\\dim\(\\mathcal\{Z\}\)\}\)random vectors; write𝒢n​\(ℋ\):=𝔼​\[𝒢^n​\(ℋ\)\]\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\):=\\mathbb\{E\}\[\\hat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\)\]for its expectation, as in the main text\. Then for anyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

\(19\)suph∈ℋ\|HSIC^​\(h​\(X\),S\)\\displaystyle\\sup\_\{h\\in\\mathcal\{H\}\}\\,\\big\|\\widehat\{\\mathrm\{HSIC\}\}\(h\(X\),S\)−HSIC\(h\(X\),S\)\|≤82ν𝒵ν𝒮log⁡\(2/δ\)n\+4​ν𝒵​ν𝒮n\\displaystyle\-\\mathrm\{HSIC\}\(h\(X\),S\)\\big\|\\;\\leq 8\\sqrt\{2\}\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\\,\\sqrt\{\\tfrac\{\\log\(2/\\delta\)\}\{n\}\}\+\\frac\{4\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\}\{n\}\+48​π​max⁡\{ν𝒵​ℓ𝒮,ν𝒮​ℓ𝒵\}​𝒢n​\(ℋ\)\.\\displaystyle\+8\\sqrt\{\\pi\}\\,\\max\\\{\\nu\_\{\\mathcal\{Z\}\}\\,\\ell\_\{\\mathcal\{S\}\},\\nu\_\{\\mathcal\{S\}\}\\,\\ell\_\{\\mathcal\{Z\}\}\\\}\\,\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)\.

###### Proof G\.4\.

Apply Corollary 23 ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\)to the two\-sample statisticγk2\\gamma\_\{k\}^\{2\}on the product RKHS with reproducing kernelk​\(\(z,s\),\(z′,s′\)\)=k𝒵​\(z,z′\)​k𝒮​\(s,s′\)k\\big\(\(z,s\),\(z^\{\\prime\},s^\{\\prime\}\)\\big\)=k\_\{\\mathcal\{Z\}\}\(z,z^\{\\prime\}\)\\,k\_\{\\mathcal\{S\}\}\(s,s^\{\\prime\}\), identified withHSIC\\mathrm\{HSIC\}viaHSIC​\(Z,S\)=γk2​\(\(Z,S\),\(Z′,S′\)\)\\mathrm\{HSIC\}\(Z,S\)=\\gamma\_\{k\}^\{2\}\(\(Z,S\),\(Z^\{\\prime\},S^\{\\prime\}\)\)for\(Z′,S′\)∼PZ⊗PS\(Z^\{\\prime\},S^\{\\prime\}\)\\sim P\_\{Z\}\\otimes P\_\{S\}\(Ni and Huo,[2024](https://arxiv.org/html/2608.10470#bib.bib27), Equation 3\)\. The bivariate function classℋ×\{id𝒮\}\\mathcal\{H\}\\times\\\{\\mathrm\{id\}\_\{\\mathcal\{S\}\}\\\}, whereid𝒮\\mathrm\{id\}\_\{\\mathcal\{S\}\}is the identity on𝒮\\mathcal\{S\}, satisfies𝔼​\[𝒢​\(\(ℋ×\{id𝒮\}\)​\(𝐗,𝐒\)\)\]≤𝔼​\[𝒢^n​\(ℋ\)\]\\mathbb\{E\}\[\\mathcal\{G\}\(\(\\mathcal\{H\}\\times\\\{\\mathrm\{id\}\_\{\\mathcal\{S\}\}\\\}\)\(\\mathbf\{X\},\\mathbf\{S\}\)\)\]\\leq\\mathbb\{E\}\[\\hat\{\\mathcal\{G\}\}\_\{n\}\(\\mathcal\{H\}\)\]by Lemma 19 ofNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27)\), since the second\-coordinate class is a singleton: under the no\-absolute\-value convention adopted byNi and Huo \([2024](https://arxiv.org/html/2608.10470#bib.bib27), Section 3\.1\),𝒢​\(\{id𝒮\}​\(𝐒\)\)=𝔼ξ​\[1n​∑i⟨ξi,Si⟩\]=0\\mathcal\{G\}\(\\\{\\mathrm\{id\}\_\{\\mathcal\{S\}\}\\\}\(\\mathbf\{S\}\)\)=\\mathbb\{E\}\_\{\\xi\}\\\!\\left\[\\frac\{1\}\{n\}\\sum\_\{i\}\\langle\\xi\_\{i\},S\_\{i\}\\rangle\\right\]=0by symmetry of the Gaussian distribution\. Substituting into the HSIC bound of Corollary 23 yields \([19](https://arxiv.org/html/2608.10470#A7.E19)\)\.

###### Corollary G\.5\(Train\-population HSIC bridge\)\.

Under the assumptions of Proposition[G\.3](https://arxiv.org/html/2608.10470#A7.Thmtheorem3), for any encoderh^∈ℋ\\hat\{h\}\\in\\mathcal\{H\}obtained fromnni\.i\.d\. training samples and anyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

\(20\)HSIC​\(h^​\(X\),S\)≤HSIC^​\(h^​\(X\),S\)\+εn​\(δ\),\\mathrm\{HSIC\}\(\\hat\{h\}\(X\),S\)\\;\\leq\\;\\widehat\{\\mathrm\{HSIC\}\}\(\\hat\{h\}\(X\),S\)\+\\varepsilon\_\{n\}\(\\delta\),where

εn​\(δ\):=8​2​ν𝒵​ν𝒮​log⁡\(2/δ\)n\+4​ν𝒵​ν𝒮n\+48​π​max⁡\{ν𝒵​ℓ𝒮,ν𝒮​ℓ𝒵\}​𝒢n​\(ℋ\)\\varepsilon\_\{n\}\(\\delta\):=8\\sqrt\{2\}\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\\sqrt\{\\frac\{\\log\(2/\\delta\)\}\{n\}\}\+\\frac\{4\\,\\nu\_\{\\mathcal\{Z\}\}\\nu\_\{\\mathcal\{S\}\}\}\{n\}\+48\\sqrt\{\\pi\}\\,\\max\\\{\\nu\_\{\\mathcal\{Z\}\}\\ell\_\{\\mathcal\{S\}\},\\nu\_\{\\mathcal\{S\}\}\\ell\_\{\\mathcal\{Z\}\}\\\}\\,\\mathcal\{G\}\_\{n\}\(\\mathcal\{H\}\)is the uniform\-concentration penalty from Proposition[G\.3](https://arxiv.org/html/2608.10470#A7.Thmtheorem3)\.

###### Proof G\.6\.

By Proposition[G\.3](https://arxiv.org/html/2608.10470#A7.Thmtheorem3), with probability at least1−δ1\-\\delta,

suph∈ℋ\|HSIC^​\(h​\(X\),S\)−HSIC​\(h​\(X\),S\)\|≤εn​\(δ\)\.\\sup\_\{h\\in\\mathcal\{H\}\}\\,\\big\|\\widehat\{\\mathrm\{HSIC\}\}\(h\(X\),S\)\-\\mathrm\{HSIC\}\(h\(X\),S\)\\big\|\\leq\\varepsilon\_\{n\}\(\\delta\)\.Specializing to the data\-dependenth^∈ℋ\\hat\{h\}\\in\\mathcal\{H\},HSIC​\(h^​\(X\),S\)−HSIC^​\(h^​\(X\),S\)≤εn​\(δ\)\\mathrm\{HSIC\}\(\\hat\{h\}\(X\),S\)\-\\widehat\{\\mathrm\{HSIC\}\}\(\\hat\{h\}\(X\),S\)\\leq\\varepsilon\_\{n\}\(\\delta\), which rearranges to \([20](https://arxiv.org/html/2608.10470#A7.E20)\)\.

## Appendix HAdditional Experiments

### H\.1Representation\-level mutual information

As a representation\-level diagnostic complementing the transfer experiment, FRHSIC tends to attain lowerMI​\(Z,S\)\\mathrm\{MI\}\(Z,S\)than Reg\-GDP at comparable prediction performance, consistent with the learned representation carrying less information aboutSSrather than only making a single trained head fair\. Figure[8](https://arxiv.org/html/2608.10470#A8.F8)reportsMI​\(Z,S\)\\mathrm\{MI\}\(Z,S\)versus prediction performance across theλ\\lambdasweep\. We treatMI​\(Z,S\)\\mathrm\{MI\}\(Z,S\)as a diagnostic, not a primary fairness audit: the KSG estimator can underestimate MI for high\-dimensional learned representations whose dependence structure violates the smoothness assumptions underlying nearest\-neighbor entropy estimation\(Gretton et al\.,[2012](https://arxiv.org/html/2608.10470#bib.bib14)\), so we interpret it together with HSIC, GDP, and downstream performance\. The adversarial methods illustrate this: ADV and LAFTR can reportMI≈0\\mathrm\{MI\}\\approx 0while their GDP remains high, which is inconsistent withMI=0⇒Z⟂S⇒GDP=0\\mathrm\{MI\}=0\\Rightarrow Z\\perp S\\Rightarrow\\mathrm\{GDP\}=0and indicates that adversarially trained representations, often concentrated on low\-dimensional manifolds, are poorly suited to nearest\-neighbor entropy estimation\. COMPAS is omitted because the KSG estimator is unreliable on its low\-dimensional, discrete\-valued features\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x8.png)Figure 8:FRHSIC yields representations that leak less aboutSS: it tends to attain lowerMI​\(Z,S\)\\mathrm\{MI\}\(Z,S\)than Reg\-GDP at comparable prediction performance, consistent with representation\-level rather than single\-head fairness\. Axes areMI​\(Z,S\)\\mathrm\{MI\}\(Z,S\)\(lower==fairer\) versus performance acrossλ\\lambda\.
### H\.2Transfer Experiment

The key advantage of FRHSIC over Reg\-GDP is*transferability*: since HSIC enforcesZ⟂SZ\\perp S, any downstream prediction head built on the learned representation is guaranteed to be fair\. To test this empirically, we train representations using each method \(atλ=10\\lambda=10\), freeze the encoder, and fit four different downstream heads on the frozen representation: linear, 2\-layer MLP, random forest, and SVM\. We then measure GDP for each head\.

Figure[9](https://arxiv.org/html/2608.10470#A8.F9)shows the results on all five datasets; we highlight two representative findings\. On Crime \(λ=100\\lambda=100\), FRHSIC attains low GDP variance across heads \(std=0\.001=0\.001\), comparable to or lower than Reg\-GDP \(0\.0020\.002\) and ADV \(0\.0050\.005\)\. On Adult \(λ=10\\lambda=10\), FRHSIC reduces cross\-head GDP variance from0\.3480\.348\(Unfair\) to0\.2440\.244, a30%30\\%reduction\. Notably, Reg\-GDP achieves lower variance \(0\.0580\.058\) on Adult but at the cost of collapsing accuracy to0\.7580\.758\(Table[1](https://arxiv.org/html/2608.10470#S6.T1)\)\. Among methods that preserve accuracy, FRHSIC provides the most consistent fairness across downstream heads\. The MI results in Table[1](https://arxiv.org/html/2608.10470#S6.T1)provide complementary evidence: MI measures the total information aboutSSinZZ, bounding the discrimination achievable by*any*downstream head \(including adversarial ones not tested here\)\. FRHSIC’s low MI \(0\.0080\.008on Adult,0\.0150\.015on Crime\) thus guarantees fairness even for worst\-case downstream use, while Reg\-GDP’s higher MI \(0\.0170\.017on Adult,0\.0560\.056on Crime\) leaves room for adversarial exploitation despite its GDP=0=0for the training head\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x9.png)Figure 9:Transfer experiment: GDP of different downstream prediction heads trained on frozen representations\. In these experiments FRHSIC maintains comparably low GDP across heads, while Reg\-GDP and FREM exhibit more head\-dependent variation\.
### H\.3Empirical Tightness of the Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)Bound

We empirically evaluate the tightness of the bound in Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)at full resolution \(m=rm=r, whereλ^m=λ^S\\hat\{\\lambda\}\_\{m\}=\\hat\{\\lambda\}\_\{S\}is the smallest positive eigenvalue andPm​𝜹f=𝜹fP\_\{m\}\\bm\{\\delta\}\_\{f\}=\\bm\{\\delta\}\_\{f\}\) on two real datasets \(Adult and Crime; we omit COMPAS because age in this dataset takes only a small number of distinct integer values, so the centered Gram matrixL~\\widetilde\{L\}onSSis near\-low\-rank andλ^S\\hat\{\\lambda\}\_\{S\}is numerically unstable\)\. For each dataset and each regularization strengthλ\\lambda, we compute \(i\) the empirical RHS‖f‖2/λ^S⋅HSIC^​\(Z,S\)\\\|f\\\|^\{2\}/\\hat\{\\lambda\}\_\{S\}\\cdot\\widehat\{\\mathrm\{HSIC\}\}\(Z,S\)on the test split, whereλ^S\\hat\{\\lambda\}\_\{S\}is the smallest positive eigenvalue ofn−1​L~n^\{\-1\}\\widetilde\{L\}, withL~=H​L​H\\widetilde\{L\}=HLHthe centered Gram matrix onSS\(eigenvalues below10−6​λ^max10^\{\-6\}\\,\\hat\{\\lambda\}\_\{\\max\}are treated as numerical zero when identifying the smallest positive eigenvalue, mirroring standard practice for truncated/regularized kernel inverses\); and \(ii\) the empirical LHS1n​∑iδ^f,i2\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{\\delta\}\_\{f,i\}^\{2\}, averaged over RKHS test functionsf​\(⋅\)=k𝒵​\(z0,⋅\)f\(\\cdot\)=k\_\{\\mathcal\{Z\}\}\(z\_\{0\},\\cdot\)for randomly sampled anchorsz0z\_\{0\}\. We plot both as a function ofλ\\lambdaon log–log axes\.

Figure[10](https://arxiv.org/html/2608.10470#A8.F10)shows the results\. The bound is valid: the empirical RHS upper\-bounds the LHS at everyλ\\lambdaon both datasets\. The vertical gap between the two curves can span several orders of magnitude and reflects the well\-known looseness of HSIC\-based bounds whenλ^S\\hat\{\\lambda\}\_\{S\}is small \(the multiplier1/λ^S1/\\hat\{\\lambda\}\_\{S\}is large becauseλ^S\\hat\{\\lambda\}\_\{S\}scales as a small fraction of the largest eigenvalue ofn−1​L~n^\{\-1\}\\widetilde\{L\}for the Gaussian kernel at the median\-heuristic bandwidth\)\. Crucially, the empirical bound remains finite and non\-vacuous — in sharp contrast with the population\-level version, where the analogous constant collapses to zero for Gaussian kernels on continuousSSand the bound becomes vacuous — so the empirical reformulation of Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)is quantitatively useful as a tractable surrogate\. Choosingm<rm<rreplacesλ^S\\hat\{\\lambda\}\_\{S\}by the largerλ^m\\hat\{\\lambda\}\_\{m\}and tightens the bound, at the cost of resolving only the top\-mmsensitive directions\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x10.png)Figure 10:Empirical tightness of the bound from Theorem[4\.12](https://arxiv.org/html/2608.10470#S4.Thmtheorem12)\. Solid line: empirical LHS1n​∑iδ^f,i2\\frac\{1\}\{n\}\\sum\_\{i\}\\hat\{\\delta\}\_\{f,i\}^\{2\}\. Dashed line: empirical RHS‖f‖2/λ^S⋅HSIC^\\\|f\\\|^\{2\}/\\hat\{\\lambda\}\_\{S\}\\cdot\\widehat\{\\mathrm\{HSIC\}\}\. Validity: the RHS upper\-bounds the LHS at everyλ\\lambda\. The vertical gap can span several orders of magnitude and reflects the looseness of HSIC\-based bounds whenλ^S\\hat\{\\lambda\}\_\{S\}is small; the bound nonetheless remains finite and non\-vacuous, in contrast to the population version, where the constant collapses to zero\.#### Bandwidth scaling ofλ^S\\hat\{\\lambda\}\_\{S\}

To validate the polynomial\-scaling claim in Remark[B\.5](https://arxiv.org/html/2608.10470#A2.Thmtheorem5), we additionally sweep the bandwidthσ∈\{0\.1,0\.3,1\.0,3\.0,10\.0\}\\sigma\\in\\\{0\.1,0\.3,1\.0,3\.0,10\.0\\\}and recordλ^S\\hat\{\\lambda\}\_\{S\}for Adult and Crime atn=1500n=1500\(Figure[11](https://arxiv.org/html/2608.10470#A8.F11)\)\. On log–log axes the curves are approximately linear, with negative slopes; the magnitudes are within a factor of two ofds=1d\_\{s\}=1, and the negative sign confirms that the centered Gram matrix onSSapproaches rank\-one asσ\\sigmagrows\. This empirical polynomial dependence is consistent with Remark[B\.5](https://arxiv.org/html/2608.10470#A2.Thmtheorem5)\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x11.png)Figure 11:Empiricalλ^S\\hat\{\\lambda\}\_\{S\}as a function of bandwidthσ\\sigmaon Adult and Crime,n=1500n=1500\. Log–log axes; the approximately linear scaling \(with empirical slopes shown in the legend\) supports the polynomial bandwidth dependence noted in Remark[B\.5](https://arxiv.org/html/2608.10470#A2.Thmtheorem5)\.

### H\.4High\-Dimensional Sensitive Attributes

A practical consequence of computing on the joint distribution rather than on conditional families is that FRHSIC tolerates increasingdim\(𝒮\)\\dim\(\\mathcal\{S\}\)gracefully, while methods built on the integral frameworkℐd\\mathcal\{I\}\_\{d\}rely internally on a kernel\-smoothed conditional weightingw^γ​\(j;i\)\\hat\{w\}\_\{\\gamma\}\(j;i\)on𝒮\\mathcal\{S\}whose statistical accuracy is subject to the curse of dimensionality of nonparametric conditional density estimation\. We illustrate the per\-iteration computational consequences on the Adult dataset by constructingSSas add\-dimensional vector from the continuous Adult features \(age, capital gain, capital loss, hours per week, plus a synthetic Gaussian feature ford=5d=5\), sweepingd∈\{1,2,3,4,5\}d\\in\\\{1,2,3,4,5\\\}, and comparing the per\-epoch training time of FRHSIC and FREM at fixedλ=10\\lambda=10and fixed bandwidths \(σS=1\.0\\sigma\_\{S\}=1\.0for FRHSIC,γ=0\.5\\gamma=0\.5for FREM\); see Figure[12](https://arxiv.org/html/2608.10470#A8.F12)\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x12.png)Figure 12:Per\-epoch training time on Adult \(n=2000n=2000\) as a function ofdim\(S\)\\dim\(S\), at fixedλ=10\\lambda=10and fixed kernel bandwidths\. FRHSIC’s HSIC loss is essentially insensitive todd\(the dimension enters only through then×nn\\times nkernel evaluation onSS\)\. FREM is consistently22–3×3\\timesslower per epoch than FRHSIC across alldd; the slowdown is dominated by the kernel\-smoothed conditional weightingw^γ​\(j;i\)\\hat\{w\}\_\{\\gamma\}\(j;i\), which evaluates add\-dimensional Gaussian kernel and an anchor\-subsampled MMD on every batch\.In our wall\-clock measurements both curves are roughly flat inddover the range we test, with FREM exhibiting a uniform∼2\\sim 2–3×3\\timesoverhead at this scale; the gap grows to roughly36×36\\timesatn=20,000n=20\{,\}000\(see Section[6\.5](https://arxiv.org/html/2608.10470#S6.SS5)\), where FREM’sO​\(n3\)O\(n^\{3\}\)scaling dominates\. Atn=2000n=2000with an anchor\-subsampled MMD \(3232anchors\), the per\-batch cost is dominated by then×nn\\times nkernel matrix on𝒵\\mathcal\{Z\}, which isdd\-independent\. The joint\-distribution route is therefore primarily a*statistical*\(rather than computational\) advantage at moderatenn: nonparametric conditional weighting on𝒮\\mathcal\{S\}inherits the slow rates ofdd\-dimensional density estimation, while the HSIC kernel\-matrix computation isdd\-agnostic\. The constant gap we observe nonetheless reflects the per\-iteration overhead of constructing conditional weights inℐd\\mathcal\{I\}\_\{d\}\-style estimators\.

### H\.5Ablation Studies

#### Kernel choice

We evaluate FRHSIC with different kernel functions fork𝒵k\_\{\\mathcal\{Z\}\}andk𝒮k\_\{\\mathcal\{S\}\}: Gaussian \(RBF\), Laplacian, and inverse multiquadric \(IMQ\)\. Table[6](https://arxiv.org/html/2608.10470#A8.T6)shows that performance is robust across kernel choices, with the Gaussian kernel performing slightly better overall\. This is consistent with the theoretical requirement that the kernels be characteristic, which all three satisfy\.

Table 6:Ablation over kernel choice \(synthetic data,λ=5\\lambda=5\)\. All kernels achieve comparable performance\.k𝒵k\_\{\\mathcal\{Z\}\}k𝒮k\_\{\\mathcal\{S\}\}AccuracyGDPGaussianGaussian0\.6200\.0015GaussianLaplacian0\.6200\.0003GaussianIMQ0\.6200\.0006LaplacianGaussian0\.6190\.0091LaplacianLaplacian0\.6200\.0001IMQGaussian0\.6200\.0002IMQIMQ0\.6200\.0027
#### Representation dimensionality

We vary the representation dimensiondZ∈\{2,8,32,64\}d\_\{Z\}\\in\\\{2,8,32,64\\\}withλ=5\\lambda=5\(Table[7](https://arxiv.org/html/2608.10470#A8.T7)\)\. All dimensions achieve comparable accuracy \(≈0\.62\\approx 0\.62\), but GDP grows with dimensionality, from0\.00010\.0001atdZ=2d\_\{Z\}=2to0\.00340\.0034atdZ=64d\_\{Z\}=64\. This confirms that higher\-dimensional representations make the HSIC penalty less effective per unit ofλ\\lambda\. The defaultdZ=8d\_\{Z\}=8provides a good balance\.

Table 7:Ablation over representation dimensionality \(synthetic data,λ=5\\lambda=5\)\.dZd\_\{Z\}AccuracyGDP20\.6200\.000180\.6200\.0001320\.6180\.0025640\.6200\.0034

### H\.6Validation\-based regularization selection

Selecting the regularization strengthλ\\lambdais a practical challenge in fair representation learning\. We use the HSIC independence test\(Gretton et al\.,[2005](https://arxiv.org/html/2608.10470#bib.bib12)\)as a validation\-based stopping criterion\. The HSIC test statistic under the null hypothesisH0:Z⟂SH\_\{0\}:Z\\perp Sfollows a weighted sum of chi\-squared variables; we use the gamma approximation\(Gretton et al\.,[2005](https://arxiv.org/html/2608.10470#bib.bib12)\), under whichn⋅HSIC^n\\cdot\\widehat\{\\mathrm\{HSIC\}\}is approximately gamma\-distributed with parameters estimated from data\.

#### Algorithm: HSIC\-test\-basedλ\\lambdaselection

1. 1\.Input:sample\{\(Xi,Si,Yi\)\}i=1n\\\{\(X\_\{i\},S\_\{i\},Y\_\{i\}\)\\\}\_\{i=1\}^\{n\}; kernelsk𝒵,k𝒮k\_\{\\mathcal\{Z\}\},k\_\{\\mathcal\{S\}\}; increasingλ\\lambdagridΛ=\{λ\(1\)<⋯<λ\(K\)\}\\Lambda=\\\{\\lambda^\{\(1\)\}<\\cdots<\\lambda^\{\(K\)\}\\\}; significance levelαtest\\alpha\_\{\\mathrm\{test\}\}\(e\.g\.,0\.050\.05\)\.
2. 2\.Foreachλ\(j\)∈Λ\\lambda^\{\(j\)\}\\in\\Lambdain increasing order: train FRHSIC atλ\(j\)\\lambda^\{\(j\)\}to obtainh^λ\(j\)\\hat\{h\}\_\{\\lambda^\{\(j\)\}\}; computeHSIC^​\(h^λ\(j\)​\(X\),S\)\\widehat\{\\mathrm\{HSIC\}\}\(\\hat\{h\}\_\{\\lambda^\{\(j\)\}\}\(X\),S\)on a held\-out validation split; run the gamma\-approximated HSIC independence test at levelαtest\\alpha\_\{\\mathrm\{test\}\}\(Gretton et al\.,[2005](https://arxiv.org/html/2608.10470#bib.bib12)\);ifthe test fails to rejectH0:Z⟂SH\_\{0\}:Z\\perp S,returnλ\(j\)\\lambda^\{\(j\)\}\.
3. 3\.Output:the smallestλ\(j\)\\lambda^\{\(j\)\}at which the test fails to reject independence, orλ\(K\)\\lambda^\{\(K\)\}if no suchλ\\lambdais found\.

This is a practical validation heuristic based on a familiar independence test, not a principled selection rule\. On large validation sets the test can be overpowered, detecting statistically significant but practically negligible dependence\. The practical\-significance variant stops at the smallestλ\(j\)\\lambda^\{\(j\)\}such thatHSIC^<ϵ\\widehat\{\\mathrm\{HSIC\}\}<\\epsilonor the test fails to reject at levelαtest\\alpha\_\{\\mathrm\{test\}\}, whereϵ\>0\\epsilon\>0is a user\-specified threshold \(e\.g\.,ϵ=10−3\\epsilon=10^\{\-3\}\), keeping the procedure useful regardless of sample size\.

We validate the procedure as follows\. For each dataset, we train FRHSIC over the grid

λ\(j\)∈\{0\.01,0\.1,0\.5,1,5,10,50,100,500\},\\lambda^\{\(j\)\}\\in\\\{0\.01,\\,0\.1,\\,0\.5,\\,1,\\,5,\\,10,\\,50,\\,100,\\,500\\\},evaluateHSIC^\\widehat\{\\mathrm\{HSIC\}\}on a held\-out validation set \(15%15\\%of data\), and apply the gamma\-approximation independence test at levelα=0\.05\\alpha=0\.05\.

Figure[13](https://arxiv.org/html/2608.10470#A8.F13)shows the results\. On Crime and COMPAS, the test successfully identifies a transition: asλ\\lambdaincreases, the HSIC test statistic falls below the rejection threshold, selectingλ≈5\\lambda\\approx 5–100100depending on the dataset and random split\. At the selectedλ\\lambda, the representation achieves low GDP while preserving prediction performance\.

On Adult \(n≈30,000n\\approx 30\{,\}000\), the test rejects independence at allλ\\lambdavalues, includingλ=500\\lambda=500where GDP is near zero\. This reflects a well\-known property of hypothesis testing: with large samples, the test detects statistically significant but*practically negligible*dependence\. In such settings, we recommend supplementing the independence test with a practical significance threshold \(e\.g\., requiringHSIC^<ϵ\\widehat\{\\mathrm\{HSIC\}\}<\\epsilonfor a user\-specifiedϵ\\epsilon\) or using the test on a smaller held\-out subsample to reduce power\.

![Refer to caption](https://arxiv.org/html/2608.10470v1/x13.png)Figure 13:Regularization strength selection via the HSIC independence test \(Appendix[H\.6](https://arxiv.org/html/2608.10470#A8.SS6)\)\. Red circles: empirical HSIC on the validation set\. Blue squares: test threshold atα=0\.05\\alpha=0\.05\. Green shading:λ\\lambdavalues where the test fails to reject independence\. The vertical dashed line marks the selectedλ\\lambda\.

## References

- Albert et al\. \(2022\)Mélisande Albert, Béatrice Laurent, Amandine Marrel, and Anouar Meynaoui\.Adaptive test of independence based on HSIC measures\.*The Annals of Statistics*, 50\(2\):858–879, 2022\.
- Angwin et al\. \(2016\)Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner\.Machine bias\.*ProPublica*, 2016\.
- Calders and Verwer \(2010\)Toon Calders and Sicco Verwer\.Three naive Bayes approaches for discrimination\-free classification\.*Data mining and knowledge discovery*, 21:277–292, 2010\.
- Cho et al\. \(2020a\)Jaewoong Cho, Gyeongjo Hwang, and Changho Suh\.A fair classifier using kernel density estimation\.In H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin, editors,*Advances in Neural Information Processing Systems*, volume 33, pages 15088–15099\. Curran Associates, Inc\., 2020a\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2020/file/ac3870fcad1cfc367825cda0101eee62\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/ac3870fcad1cfc367825cda0101eee62-Paper.pdf)\.
- Cho et al\. \(2020b\)Jaewoong Cho, Gyeongjo Hwang, and Changho Suh\.A fair classifier using mutual information\.In*2020 IEEE International Symposium on Information Theory \(ISIT\)*, pages 2521–2526, 2020b\.
- Cover and Thomas \(2006\)Thomas M\. Cover and Joy A\. Thomas\.*Elements of Information Theory*\.Wiley\-Interscience, 2 edition, 2006\.
- Creager et al\. \(2019\)Elliot Creager, David Madras, Jörn\-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel\.Flexibly fair representation learning by disentanglement\.In*International conference on machine learning*, pages 1436–1445\. PMLR, 2019\.
- Deka and Sutherland \(2023\)Namrata Deka and Danica J Sutherland\.MMD\-B\-Fair: Learning fair representations with statistical testing\.In*International Conference on Artificial Intelligence and Statistics*, pages 9564–9576\. PMLR, 2023\.
- Ding et al\. \(2021\)Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt\.Retiring adult: New datasets for fair machine learning\.In*Advances in Neural Information Processing Systems*, volume 34, pages 6478–6490, 2021\.
- Fukumizu et al\. \(2007\)Kenji Fukumizu, Francis R\. Bach, and Arthur Gretton\.Statistical consistency of kernel canonical correlation analysis\.*Journal of Machine Learning Research*, 8:361–383, 2007\.
- Grari et al\. \(2022\)Vincent Grari, Sylvain Lamprier, and Marcin Detyniecki\.Fairness without the sensitive attribute via causal variational autoencoder\.In*International Joint Conference on Artificial Intelligence*, 2022\.
- Gretton et al\. \(2005\)Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf\.Measuring statistical dependence with Hilbert\-Schmidt norms\.In*Algorithmic Learning Theory: 16th International Conference, ALT 2005, Singapore, October 8\-11, 2005\. Proceedings 16*, pages 63–77\. Springer, 2005\.
- Gretton et al\. \(2007\)Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, and Alex Smola\.A kernel statistical test of independence\.*Advances in neural information processing systems*, 20, 2007\.
- Gretton et al\. \(2012\)Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola\.A kernel two\-sample test\.*The Journal of Machine Learning Research*, 13\(1\):723–773, 2012\.
- Jiang et al\. \(2022\)Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Ali Mostafavi, and Xia Hu\.Generalized demographic parity for group fairness\.In*International Conference on Learning Representations*, 2022\.
- Kallenberg \(2002\)Olav Kallenberg\.*Foundations of Modern Probability*\.Springer, 2 edition, 2002\.
- Koltchinskii and Giné \(2000\)Vladimir Koltchinskii and Evarist Giné\.Random matrix approximation of spectra of integral operators\.*Bernoulli*, 6\(1\):113–167, 2000\.
- Kong et al\. \(2025\)Insung Kong, Kunwoong Kim, and Yongdai Kim\.Fair representation learning for continuous sensitive attributes using expectation of integral probability metrics\.*IEEE transactions on pattern analysis and machine intelligence*, 2025\.
- Kusner et al\. \(2017\)Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva\.Counterfactual fairness\.In*Advances in Neural Information Processing Systems*, volume 30, 2017\.
- Li et al\. \(2022\)Zhu Li, Adrián Pérez\-Suay, Gustau Camps\-Valls, and Dino Sejdinovic\.Kernel dependence regularizers and Gaussian processes with applications to algorithmic fairness\.*Pattern Recognition*, 132:108922, 2022\.
- Louizos et al\. \(2016\)Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel\.The variational fair autoencoder\.In*International Conference on Learning Representations*, 2016\.
- Ma et al\. \(2020\)Wan\-Duo Kurt Ma, JP Lewis, and W Bastiaan Kleijn\.The HSIC bottleneck: Deep learning without back\-propagation\.In*Proceedings of the AAAI conference on artificial intelligence*, volume 34, pages 5085–5092, 2020\.
- Ma et al\. \(2018\)Zhengming Ma, Zengrong Zhan, Xiaoyuan Ouyang, and Xue Su\.Nonlinear dimensionality reduction based on HSIC maximization\.*IEEE Access*, 6:55537–55555, 2018\.
- Madras et al\. \(2018\)David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel\.Learning adversarially fair and transferable representations\.In*International Conference on Machine Learning*, pages 3384–3393\. PMLR, 2018\.
- Mary et al\. \(2019\)Jean Mary, Clément Calauzènes, and Noureddine El Karoui\.Fairness\-aware learning for continuous attributes and treatments\.In*International Conference on Machine Learning*, pages 4382–4391, 2019\.
- Nadaraya \(1964\)Elizbar A Nadaraya\.On estimating regression\.*Theory of Probability & Its Applications*, 9\(1\):141–142, 1964\.
- Ni and Huo \(2024\)Yijin Ni and Xiaoming Huo\.A uniform concentration inequality for kernel\-based two\-sample statistics\.*arXiv preprint arXiv:2405\.14051*, 2024\.
- Pearl \(2009\)Judea Pearl\.*Causality: Models, Reasoning, and Inference*\.Cambridge University Press, 2 edition, 2009\.
- Pérez\-Suay et al\. \(2017\)Adrián Pérez\-Suay, Valero Laparra, Gonzalo Mateo\-García, Jordi Muñoz\-Marí, Luis Gómez\-Chova, and Gustau Camps\-Valls\.Fair kernel learning\.In*Machine Learning and Knowledge Discovery in Databases \(ECML PKDD\)*, pages 339–355\. Springer, 2017\.
- Quadrianto et al\. \(2019\)Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas\.Discovering fair representations in the data domain\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2019\.
- Romano et al\. \(2020\)Yaniv Romano, Stephen Bates, and Emmanuel J Candès\.Achieving equalized odds by resampling sensitive attributes\.In*Advances in Neural Information Processing Systems*, volume 33, pages 361–371, 2020\.
- Sejdinovic et al\. \(2013\)Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu\.Equivalence of distance\-based and RKHS\-based statistics in hypothesis testing\.*The annals of statistics*, pages 2263–2291, 2013\.
- Song et al\. \(2007\)Le Song, Alex Smola, Arthur Gretton, Karsten M Borgwardt, and Justin Bedo\.Supervised feature selection via dependence estimation\.In*Proceedings of the 24th international conference on Machine learning*, pages 823–830, 2007\.
- Song et al\. \(2012\)Le Song, Alex Smola, Arthur Gretton, Justin Bedo, and Karsten Borgwardt\.Feature selection via dependence maximization\.*The Journal of Machine Learning Research*, 13\(1\):1393–1434, 2012\.
- Szabó and Sriperumbudur \(2018\)Zoltán Szabó and Bharath K Sriperumbudur\.Characteristic and universal tensor product kernels\.*Journal of Machine Learning Research*, 18\(233\):1–29, 2018\.
- Székely et al\. \(2007\)Gábor J Székely, Maria L Rizzo, and Nail K Bakirov\.Measuring and testing dependence by correlation of distances\.*The annals of statistics*, 35\(6\):2769–2794, 2007\.
- Watson \(1964\)Geoffrey S Watson\.Smooth regression analysis\.*Sankhyā: The Indian Journal of Statistics, Series A*, pages 359–372, 1964\.
- Zemel et al\. \(2013\)Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork\.Learning fair representations\.In Sanjoy Dasgupta and David McAllester, editors,*Proceedings of the 30th International Conference on Machine Learning*, volume 28 of*Proceedings of Machine Learning Research*, pages 325–333, Atlanta, Georgia, USA, 17–19 Jun 2013\. PMLR\.URL[https://proceedings\.mlr\.press/v28/zemel13\.html](https://proceedings.mlr.press/v28/zemel13.html)\.