Understanding Backdoor Vulnerabilities in Vertical Federated Learning: The Gap Between Research and Practice

# Understanding Backdoor Vulnerabilities in Vertical Federated Learning:The Gap Between Research and Practice
Source: [https://arxiv.org/html/2608.12962](https://arxiv.org/html/2608.12962)
Jialin LuJunjie ShanJunyuan ZhangShuya YangKa\-Ho ChowAffiliation:School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China

###### Abstract

Vertical Federated Learning \(VFL\) enables organizations holding complementary features of shared entities to collaborate and train models\. In this setting, the initiator can withhold information about the learning task, while other contributors participate without exposing their local datasets, creating an asymmetric information structure aligned with growing privacy demands\. However, this asymmetry is a double\-edged sword\. Among various threats, backdoor attacks are particularly concerning because VFL not only enables malicious contributors to poison the model during training, but also allows them to activate the backdoor at inference time to manipulate predictions\. Although prior work has reported near\-perfect attack success rates and proposed effective defenses, we find that most findings fail to hold under realistic conditions, exposing a fundamental gap between research and practice\. In this paper, we present a systematic, practice\-oriented study of backdoor vulnerabilities in VFL, revealing this gap in both methodological design and evaluation practices\. We show that existing approaches overlook key practical constraints and therefore rely on unrealistic prior knowledge\. Furthermore, these limitations have remained hidden due to poorly designed evaluation practices in the literature\. To bridge this gap, we redefine threat models under realistic constraints, propose practical backdoor workflows, and introduce BVBench, a backdoor\-centric benchmark that enables fair, practical, and comprehensive evaluation, preloaded with state\-of\-the\-art baselines\. BVBench provides strong evidence of the fragility of the current understanding of VFL backdoor risks and establishes a foundation for steering research toward uncovering practical vulnerabilities and developing more meaningful defenses\.

## IIntroduction

Cross\-organizational collaboration is key to unlocking the full potential of modern machine learning \(ML\), which thrives on rich and expressive data to achieve high predictive performance\. In practice, organizations across domains often possess complementary features about the same set of entities, such as credit history in financial institutions\[[45](https://arxiv.org/html/2608.12962#bib.bib22)\]and health conditions in healthcare providers\[[30](https://arxiv.org/html/2608.12962#bib.bib17),[15](https://arxiv.org/html/2608.12962#bib.bib23)\]\. Integrating these distributed features can enhance data expressiveness and improve predictive performance\. However, despite enabling multi\-billion\-dollar opportunities, such collaborations remain difficult due to data privacy regulations\[[9](https://arxiv.org/html/2608.12962#bib.bib19),[3](https://arxiv.org/html/2608.12962#bib.bib20)\]that prohibit sharing raw data across organizational boundaries\.

Vertical Federated Learning \(VFL\)\[[30](https://arxiv.org/html/2608.12962#bib.bib17),[20](https://arxiv.org/html/2608.12962#bib.bib21),[43](https://arxiv.org/html/2608.12962#bib.bib15),[16](https://arxiv.org/html/2608.12962#bib.bib36)\]has emerged as a promising solution to this challenge\. As illustrated in Figure[1](https://arxiv.org/html/2608.12962#S1.F1), an insurance company \(the “active” party\) can collaborate with a wearable provider and a healthcare center \(the “passive” parties\)\. When a potential client applies for insurance, the active party requests the passive parties to identify matching records, process them locally through private feature extractors, and return the resulting embeddings\.

![Refer to caption](https://arxiv.org/html/2608.12962v1/teaser.png)Fig\. 1:VFL enables an active party \(e\.g\., an insurance company\) to collaborate with passive parties \(e\.g\., a wearable provider and a healthcare center\) and leverage complementary features of shared entities to improve ML tasks, such as underwriting risk assessment, without sharing raw data\. However, the inherent information asymmetry in VFL can be exploited: a malicious participant \(e\.g\., the wearable provider\) may implant a backdoor during training and later manipulate model predictions at inference time\.These embeddings, which are not human\-interpretable, are aggregated by the active party and used by a classifier to predict underwriting risk\. In this way, predictions can leverage sensitive information, such as health status and activity data, without exposing raw data\. This capability is enabled by jointly training models across parties while preserving data locality and creating information asymmetry: the active party knows the ML task but not the raw data, whereas passive parties know their local data but not the task itself\. While this asymmetry makes cross\-organizational learning practical, it also introduces unique security risks\.

Backdoor attacks\[[40](https://arxiv.org/html/2608.12962#bib.bib13),[2](https://arxiv.org/html/2608.12962#bib.bib1),[18](https://arxiv.org/html/2608.12962#bib.bib8)\]are a particularly concerning threat in VFL\. In such attacks, an adversary manipulates the training process to implant a hidden association between a trigger pattern and a target prediction\. At inference time, the presence of the trigger causes the model to produce adversary\-chosen outputs while behaving normally otherwise\. VFL provides a unique and favorable setting for such attacks: passive parties participate in both training, where backdoors can be implanted, and inference, where they can be activated\. For example, in Figure[1](https://arxiv.org/html/2608.12962#S1.F1), a malicious wearable provider could implant a backdoor during training and later inject trigger patterns into embeddings at inference time, overriding the classifier’s decision \(e\.g\., approving an otherwise ineligible insurance application\)\. Such scenarios raise serious concerns about the security of VFL systems and have motivated a growing body of research on attack and defense mechanisms\. However, do these studies truly reflect how VFL backdoors behave in real\-world deployments?

In this paper, we argue that the current understanding of backdoor vulnerabilities in VFL is fragile\. Despite rapid research progress, it remains unclear to what extent existing findings generalize to practice\. This gap has important consequences: practitioners may underestimate risks when systems appear robust, or overtrust defenses that fail under realistic conditions\. By systematizing existing work, we identify a disconnect between how VFL backdoor vulnerabilities are studied and how VFL systems operate in practice\. We attribute this gap to misalignment along two dimensions: \(i\) methodological design and \(ii\) evaluation design\.

Methodological Design\.Existing attacks and defenses often rely on unrealistic assumptions regarding threat models and, consequently, problem formulation\. For example, many attacks assume access to seemingly obtainable knowledge of the ML task \(e\.g\., class semantics\), even though the active party has no obligation to share such information\. In practice, a malicious passive party is unlikely to know the task to which it contributes, including the number of classes, let alone their meanings\. As shown in Figure[2](https://arxiv.org/html/2608.12962#S1.F2)\(orange\), relaxing this assumption significantly reduces the success rate of state\-of\-the\-art attacks\. This finding suggests that prior work may fail to generalize to real deployments and overestimate attack effectiveness, highlighting the need to ground future research in more realistic assumptions\.

Fig\. 2:The high attack success rates \(red\) reported in prior work can be misleading\. Existing approaches often rely on assumptions that are unlikely to hold in practice \(orange\) or are evaluated under unrealistic experimental settings \(blue\)\.Evaluation Design\.The evaluation of VFL backdoor vulnerabilities is fragmented\. Existing studies rely on unrealistic datasets, inconsistent experimental setups, and limited metrics that fail to capture critical properties\. More importantly, attack and defense performance is highly sensitive to these choices\. As shown in Figure[2](https://arxiv.org/html/2608.12962#S1.F2)\(blue\), attacks reported to achieve near\-perfect success rates can only reach a meaningless level of effectiveness on realistic VFL datasets, an effect previously overlooked due to the lack of standardized benchmarks and reproducible implementations\. Overall, current evaluations do not provide a reliable or objective understanding of practical security risks\.

As interest in VFL continues to grow in both academia and industry, the absence of a practice\-oriented framework risks fostering misconceptions and hindering progress\. To address this gap, we make the following contributions:

- •Knowledge Systematization\.We provide a structured analysis of backdoor attacks and defenses, uncover their limitations, redefine a threat model grounded in realistic assumptions, and identify a practical backdoor workflow\.
- •Evaluation Framework\.We develop BVBench, a VFL backdoor\-centric benchmark that includes realistic datasets, comprehensive metrics, and a standardized evaluation recipe for fair and reproducible evaluations\.
- •Empirical Studies\.We evaluate existing methods under BVBench and show that they exhibit limited practicality and are highly sensitive to evaluation choices\.

Our work benefits multiple stakeholders\. For researchers, it clarifies realistic threat models and highlights open challenges under a practical backdoor workflow\. For practitioners, it provides new understanding of risks and offers tools and guidelines for rigorous evaluation\. Our artifacts are[here](https://github.com/HKU-TASR/BVBench)\.

## IIBackground

### II\-AVertical Federated Learning

Although VFL supports various model families, neural network\-based VFL\[[21](https://arxiv.org/html/2608.12962#bib.bib41),[41](https://arxiv.org/html/2608.12962#bib.bib35),[28](https://arxiv.org/html/2608.12962#bib.bib26),[34](https://arxiv.org/html/2608.12962#bib.bib42),[33](https://arxiv.org/html/2608.12962#bib.bib43),[1](https://arxiv.org/html/2608.12962#bib.bib44),[44](https://arxiv.org/html/2608.12962#bib.bib16)\]has been the primary focus of recent research due to its strong representational power\. We therefore focus on this setting\. Figure[3](https://arxiv.org/html/2608.12962#S2.F3)illustrates a system with one active party andNNpassive parties holding complementary features for shared entities\. VFL consists of three stages:

![Refer to caption](https://arxiv.org/html/2608.12962v1/vfl-internals.png)Fig\. 3:Information asymmetry is fundamental to VFL\. During the forward pass \(green\), passive parties send only embeddings to the active party, which aggregates them to produce predictions\. During the backward pass \(red\), the active party returns gradients with respect to each party’s embeddings\. As a result, the active party has access to the task and labels but not the raw data, while passive parties retain raw data but have no visibility into the task or labels\.Stage 1: Negotiation\.The active party defines the ML task and collects labels for existing entities to be used for training and evaluation\. Because it may lack sufficient features, or even any features at all, the active party recruits passive parties to contribute complementary data\. The parties negotiate incentives and agree on system design choices \(e\.g\., model architectures and optimization strategies\), as well as participation in both training and inference\.

Stage 2: Training\.Training begins with privacy\-preserving entity alignment across parties, such as Private Set Intersection \(PSI\)\[[24](https://arxiv.org/html/2608.12962#bib.bib34),[42](https://arxiv.org/html/2608.12962#bib.bib38),[38](https://arxiv.org/html/2608.12962#bib.bib37)\]\. At each iteration, the active party selects a minibatch of sample IDs and distributes them to passive parties \(we use batch size11for brevity\)\.

- •*Forward Pass*: As shown in Figure[3](https://arxiv.org/html/2608.12962#S2.F3)\(green path\), each passive partyiiretrieves its local data𝒙i\\bm\{x\}\_\{i\}and feeds it into its bottom modelℱiB\\mathcal\{F\}^\{B\}\_\{i\}to produce an embedding𝓔i\\bm\{\\mathcal\{E\}\}\_\{i\}, which is then sent to the active party\. The active party concatenates these embeddings𝓔1⊕⋯⊕𝓔N\\bm\{\\mathcal\{E\}\}\_\{1\}\\oplus\\cdots\\oplus\\bm\{\\mathcal\{E\}\}\_\{N\}and applies the top modelℱT\\mathcal\{F\}^\{T\}as a classifier to generate predictiony^\\hat\{y\}\.
- •*Backward Pass*: As shown in Figure[3](https://arxiv.org/html/2608.12962#S2.F3)\(red path\), the active party computes the loss using predicted labely^\\hat\{y\}and ground\-truth labelyy, and updates the top modelℱT\\mathcal\{F\}^\{T\}via gradient descent\. By the chain rule, it then computes and sends gradients w\.r\.t\. each𝓔i\\bm\{\\mathcal\{E\}\}\_\{i\}to the corresponding passive partyii, which continues backpropagation to update its bottom modelℱiB\\mathcal\{F\}^\{B\}\_\{i\}\. Overall, all models \(ℱT,ℱ1B,…,ℱNB\\mathcal\{F\}^\{T\},\\mathcal\{F\}^\{B\}\_\{1\},\.\.\.,\\mathcal\{F\}^\{B\}\_\{N\}\) are jointly optimized to improve prediction accuracy\.

This process continues until convergence or until a predefined number of epochs is reached\.

Stage 3: Inference\.In VFL, all parties must participate during inference\. Upon receiving a query, the active party broadcasts the sample ID, requests embeddings from passive parties, and performs the forward pass described above\.

### II\-BBackdoor Vulnerabilities

Attacks\.Backdoor attacks manipulate model behavior by implanting a hidden trigger during training that induces a target prediction at inference time\. This is achieved by injecting a trigger pattern into a subset of training samples, allowing the model to associate the trigger with the desired output\. A backdoor attack aims to preserve utility on clean inputs, induce the target output when the trigger is present, and remain difficult to detect during both the training\-time manipulation and inference\-time injection\. While backdoor attacks have been extensively studied in centralized learning and horizontal federated learning \(HFL\)\[[13](https://arxiv.org/html/2608.12962#bib.bib27),[7](https://arxiv.org/html/2608.12962#bib.bib28)\], where clients hold disjoint samples with shared features, VFL introduces distinct opportunities and challenges\.

- •*Opportunities*: VFL requires passive parties to participate in training and inference and to communicate through embeddings, creating two advantages for the adversaryii\. \(i\) Triggers can be crafted and injected directly in the embedding space \(i\.e\.,𝓔i\\bm\{\\mathcal\{E\}\}\_\{i\}\) rather than the input space \(i\.e\.,𝒙i\\bm\{x\}\_\{i\}\)\. This avoids input\-space constraints \(e\.g\., valid image pixel ranges\)\. \(ii\) A malicious passive party has consistent control over part of the top model input, eliminating the need for extra mechanisms to inject triggers\.
- •*Challenges*: Passive parties cannot observe ground\-truth labels\[[10](https://arxiv.org/html/2608.12962#bib.bib5),[12](https://arxiv.org/html/2608.12962#bib.bib6)\]\. This complicates establishing the association between triggers and target outputs, which in non\-VFL settings is typically achieved by consistently injecting triggers into training samples of the target class and allowing the model to learn this shortcut\. In addition, only a small fraction of the entire model is accessible to the adversary \(i\.e\., its own bottom modelℱiB\\mathcal\{F\}^\{B\}\_\{i\}\), making it harder to control both attack effectiveness and stealth\.

Defenses\.Backdoor defenses aim to preserve model utility while preventing trigger\-induced misbehavior\. In VFL, defenses are the responsibility of the active party, which orchestrates the system and produces predictions for downstream use\. Although a wide range of defenses has been proposed in centralized and HFL settings\[[31](https://arxiv.org/html/2608.12962#bib.bib29),[29](https://arxiv.org/html/2608.12962#bib.bib31),[19](https://arxiv.org/html/2608.12962#bib.bib30),[23](https://arxiv.org/html/2608.12962#bib.bib33),[11](https://arxiv.org/html/2608.12962#bib.bib32)\], VFL presents distinct opportunities and challenges\.

- •*Opportunities*: The active party observes embeddings from all passive parties \(𝓔1,…,𝓔N\\bm\{\\mathcal\{E\}\}\_\{1\},\.\.\.,\\bm\{\\mathcal\{E\}\}\_\{N\}\), while each passive partyiionly observes its own contributions \(i\.e\.,𝓔i\\bm\{\\mathcal\{E\}\}\_\{i\}\)\. This asymmetry enables cross\-party consistency checks and the potential identification of anomalous embeddings\.
- •*Challenges*: Because all parties are required during inference, excluding a suspected malicious party is impractical, as the entire system typically needs to be reconstructed from scratch\. Hence, unlike non\-VFL settings, mitigating backdoor effects is more preferred than detection alone\.

In summary, VFL introduces unique structural constraints that fundamentally shape the design of attacks and defenses\. Next, we systematically analyze existing approaches \(Sections[III](https://arxiv.org/html/2608.12962#S3)and[IV](https://arxiv.org/html/2608.12962#S4)\) and identify key limitations in current evaluation practices, motivating the need for a comprehensive benchmark \(Section[V](https://arxiv.org/html/2608.12962#S5)\)\.

TABLE I:Current VFL backdoor attacks exhibit practicality gaps in threat model and their current designs are either at odds with or poorly aligned with the practical backdoor workflow outlined in Section[III\-B](https://arxiv.org/html/2608.12962#S3.SS2)\.

## IIISystematizing VFL Backdoor Attacks

Current backdoor attacks share a similar workflow\. First, they collect or infer labels for some training samples\. Then, they craft a trigger pattern and use those labeled samples to establish a shortcut between this trigger and the target class\. We systematize existing works through the lens of practicality in the threat model, attack design, and stealthiness\.

### III\-AThreat Model

Threat models define the conditions under which an attack is launched\. Ensuring practicality is critical, as it determines whether the insights reflect real\-world vulnerabilities\. We summarize existing threat models as follows:

#### III\-A1Attacker’s Goal

The adversary \(a passive partyii\) constructs a trigger injection functionℐ⁡\(ℱiB,𝒙i,t\)\\mathcal\{I\}\(\\mathcal\{F\}\_\{i\}^\{B\},\\bm\{x\}\_\{i\},t\)for a designated target classttsuch that, when activated during inference, the prediction by the top modelℱT​\(𝓔1⊕⋯⊕ℐ⁡\(ℱiB,𝒙i,t\)⊕⋯⊕𝓔N\)=t\\mathcal\{F\}^\{T\}\(\\bm\{\\mathcal\{E\}\}\_\{1\}\\oplus\\cdots\\oplus\\mathcal\{I\}\(\\mathcal\{F\}\_\{i\}^\{B\},\\bm\{x\}\_\{i\},t\)\\oplus\\cdots\\oplus\\bm\{\\mathcal\{E\}\}\_\{N\}\)=t\. When the adversary submits benign embeddings produced by its bottom modelℱiB\\mathcal\{F\}^\{B\}\_\{i\}\(i\.e\.,ℱiB​\(𝒙i\)\\mathcal\{F\}^\{B\}\_\{i\}\(\\bm\{x\}\_\{i\}\)\), the prediction should maintain normal accuracy\. This behavior is achieved by poisoning the VFL training process while remaining undetectable by the active party\.

#### III\-A2Attacker’s Capabilities

Most existing attacks assume capabilities consistent with those of a passive party in standard VFL\. During training, the adversary can submit arbitrary embeddings to the active party and manipulate received gradients to update its bottom model\. During inference, the adversary can submit trigger\-injected embeddings to activate the backdoor and influence predictions\.

#### III\-A3Attacker’s Knowledge

The main distinction across existing attacks lies in their assumptions about prior knowledge, as summarized in Table[I](https://arxiv.org/html/2608.12962#S2.T1)\(Columns 2–3\)\. The purpose of prior knowledge is to help acquire labels of training samples, which are essential for the attacker to craft triggers and establish the connection between the trigger and the target class\. Current works make various assumptions, including:

- •Target\-Class Labelℒt​a​r​g​e​t\\mathcal\{L\}\_\{\{target\}\}\.All attacks require the adversary to be able to identify or infer some training samples belonging to the target class\. For example, VILLAIN\[[2](https://arxiv.org/html/2608.12962#bib.bib1)\]and BadVFL∗\[[40](https://arxiv.org/html/2608.12962#bib.bib13)\]require one such sample, while BackSplitVFL\[[14](https://arxiv.org/html/2608.12962#bib.bib7)\]and LMP\[[18](https://arxiv.org/html/2608.12962#bib.bib8)\]require multiple\. This requirement is unlikely to be met because the active party is not obligated to disclose task\-specific information\. Passive participants do not even know about the semantics of the classes, let alone identify a target class for attack\.
- •Full Label Spaceℒf​u​l​l\\mathcal\{L\}\_\{full\}\.Some attacks, such as BadVFL\[[22](https://arxiv.org/html/2608.12962#bib.bib9)\], further assume knowledge of the entire label space, including labeled samples for all classes or the total number of classes\. For the same reason, this is even more unlikely to be accessible than the target\-class label alone\.
- •Top Model Posterior𝒫\\mathcal\{P\}\.Some attacks assume the adversary can observe the top model’s posterior during inference, which will be used to infer labels and optimize the trigger\. This is in direct conflict with the VFL setting where the passive parties have no access to the top model\.

In short, the information asymmetry in VFL makes such task\-related knowledge inaccessible\. We therefore redefine a practical threat model for VFL backdoor attacks as follows:

Practical Threat Model:The attacker possessing the capabilities as a passive party \(§[III\-A2](https://arxiv.org/html/2608.12962#S3.SS1.SSS2)\) should achieve the attack goal \(§[III\-A1](https://arxiv.org/html/2608.12962#S3.SS1.SSS1)\) without prior knowledge of the ML task, such as class semantics, target\-class labels, full label space, or any other task\-related information\.Under this threat model, no existing attacks remain feasible\.

### III\-BAttack Design

Based on the redefined threat model, we outline a practical backdoor workflow as follows:

Practical Backdoor Workflow:First, label acquisition should aim to cluster training samples of the same class without access to task\-related knowledge\. Second, each cluster should be assigned a pseudo\-label and a dedicated trigger pattern, and the class semantics of each pseudo\-label should be inferred after deployment by sending trigger\-injected queries and observing responses\. Third, backdoor learning should be formulated as a multi\-target instance while balancing effectiveness across targets\.Next, we show that current attack designs fall short in aligning with this workflow\.

#### III\-B1Label Acquisition

Existing attacks rely on inaccessible labeled samples\. Most of them justify this assumption by requiring only a few such samples and inferring the rest\. They either exploit the gradients from the active party or the embeddings produced by the bottom model \(Columns 4–5, Table[I](https://arxiv.org/html/2608.12962#S2.T1)\), but such signals are unstable across training\. As shown in Figure[4](https://arxiv.org/html/2608.12962#S3.F4), the former is informative only in early training, while the latter remains useless until the bottom model begins to extract useful embeddings\. In practice, learning progress depends on uncontrollable factors such as task complexity\. As analyzed in Section[VI](https://arxiv.org/html/2608.12962#S6), label quality plays a decisive role in backdoor attacks\. Robust label acquisition from these unstable signals remains overlooked\.

![Refer to caption](https://arxiv.org/html/2608.12962v1/attack-label-gradients.png)\(a\)Gradients
![Refer to caption](https://arxiv.org/html/2608.12962v1/attack-label-embeddings.png)\(b\)Embeddings

Fig\. 4:Label inference can be unstable\. The cluster effect of gradients and embeddings varies across training progress\.
#### III\-B2Trigger Design for Multi\-Target Learning

Multi\-target learning poses a significant challenge to trigger design for three reasons \(Columns 6–8, Table[I](https://arxiv.org/html/2608.12962#S2.T1)\)\. First, each pseudo\-label requires a distinct trigger pattern and/or position, while some existing methods \(VILLAIN\[[2](https://arxiv.org/html/2608.12962#bib.bib1)\], BadVFL∗\[[40](https://arxiv.org/html/2608.12962#bib.bib13)\]and LFBA\[[27](https://arxiv.org/html/2608.12962#bib.bib11)\]\) use fixed and class\-agnostic triggers, so the trigger pattern and injected position are the same for any target class\. Second, it is non\-trivial to scale to multiple targets even if trigger designs are class\-aware because of potential interference\. As shown in Figure[5](https://arxiv.org/html/2608.12962#S3.F5), the coexistence of certain triggers can lead to mild \(orange\) or even drastic \(green\) degradation in attack effectiveness\. Third, as more shortcuts need to be memorized, the trigger needs to produce stronger signals for the top model to discover\. While our empirical analysis found that embedding\-space triggers can be more effective, current methods provide no mechanism for balancing the attack strength across different classes, which can lead to cross\-target conflicts and instability\.

Fig\. 5:Trigger interference can hinder attack effectiveness in multi\-target settings\.

### III\-CStealthiness Consideration

Current stealthiness definitions are oversimplified and under\-defined\. Existing definitions rely on simple heuristics, such as repetitive patterns in the embedding space\[[14](https://arxiv.org/html/2608.12962#bib.bib7)\], or an abnormal range in the input or embedding space\[[14](https://arxiv.org/html/2608.12962#bib.bib7),[2](https://arxiv.org/html/2608.12962#bib.bib1)\]\. We argue that stealthiness should be defined from the defender’s perspective and with the defender’s knowledge, while remaining assessable under the attacker’s knowledge and capabilities\. Since embeddings are used in both training and inference and are accessible to both active and passive parties, they are the natural medium for defining stealthiness: if poisoned samples are easily identifiable in the embedding space, then the attack is not stealthy\. Based on this intuition, we find that only BackSplitVFL\[[14](https://arxiv.org/html/2608.12962#bib.bib7)\]and VILLAIN\[[2](https://arxiv.org/html/2608.12962#bib.bib1)\]have relevant embedding\-space constraints \(Columns 9–10, Table[I](https://arxiv.org/html/2608.12962#S2.T1)\)\. That said, these constraints cannot hide trigger\-injected embeddings from defenders\. For example, as shown in Figure[6](https://arxiv.org/html/2608.12962#S3.F6), BackSplitVFL uses maximum per\-dimension and overall norm of clean embeddings to constrain poisoned embeddings, but such a value is already highly anomalous; VILLAIN uses a trigger pattern based on the averaged standard deviation over dimensions that have high standard deviation, but adding such triggers can still lead to off\-distribution embeddings\. Furthermore, the defender can access all parties’ embeddings, which can make the attack even easier to detect if the defender leverages cross\-party consistency checks\.

Fig\. 6:Existing stealthiness constraints do not help attacks remain under the radar\. For instance, BackSplitVFL\[[14](https://arxiv.org/html/2608.12962#bib.bib7)\]constrains trigger norm to be within the maximum norm of clean embeddings, but it is still highly anomalous and can be easily detected using the 99% quantile\.Practicality Insights:Current stealthiness definitions are fragile and poorly aligned with defender’s knowledge\. It should be defender\-driven, attacker\-assessible, defined in the embedding space, and tight enough to make malicious embeddings indistinguishable from benign ones\.

## IVSystematizing VFL Backdoor Defenses

TABLE II:Current VFL backdoor defenses exhibit practicality gaps in prior knowledge, design, and recovery quality\.Backdoor defenses in VFL can be applied at both train time and test time, with three objectives: \(i\) mitigation, which reduces the attack influence during training; \(ii\) recovery, which aims to restore the original prediction of attacked samples; and \(iii\) detection, which identifies attacked samples\. We systematize existing works through the lens of the defender’s knowledge, defense design, and the overlooked issue of post\-attack utility recovery\.

### IV\-AThreat Model

#### IV\-A1Defender’s Goal

The defender, acting as the active party, aims to prevent backdoor attacks from malicious passive parties and preserve clean model utility simultaneously\. This can be achieved in three forms: \(i\) train\-time mitigation; \(ii\) test\-time recovery; and \(iii\) test\-time detection\.

#### IV\-A2Defender’s Capabilities

Defenders’ capabilities are consistent with those of the active party\. During training, the defender can inspect, manipulate and store embeddings from all passive parties before feeding them into the top model\. The gradients derived from the top model can also be arbitrarily manipulated before sending back to passive parties\. During inference, the defender can also manipulate embeddings and the top model\.

#### IV\-A3Defender’s Knowledge

VFL\-specific defenses require some prior knowledge: \(i\) clean reference embeddings and \(ii\) knowledge of the adversarial environment such as the number of attackers and target classes, as summarized in Table[II](https://arxiv.org/html/2608.12962#S4.T2)\(a\) \(Columns 2–3\)\. Obviously, these assumptions are unlikely to hold, and hence we redefine a practical threat model for VFL backdoor defenses as follows:

Practical Threat Model:The defender possessing the capabilities as an active party \(§[IV\-A2](https://arxiv.org/html/2608.12962#S4.SS1.SSS2)\) should achieve the defense goal \(§[IV\-A1](https://arxiv.org/html/2608.12962#S4.SS1.SSS1)\) without clean reference embeddings and the adversarial environment \(§[IV\-A3](https://arxiv.org/html/2608.12962#S4.SS1.SSS3)\)\.Under this threat model, no existing VFL\-specific defenses remain practical, as they all explicitly or implicitly rely on clean reference embeddings\.

### IV\-BDefense Design

#### IV\-B1Proactive Mitigation

Our empirical analysis in Section[VII](https://arxiv.org/html/2608.12962#S7)shows the clear advantage of VFL\-specific defenses over generic ones, yet only a few methods adopt a proactive approach \(Column 4, Table[II](https://arxiv.org/html/2608.12962#S4.T2)\)\. Proactive mitigation before inference deserves more attention, as it reduces the inference burden and better preserves efficiency\.

#### IV\-B2Multi\-target Compatibility

Similar to attacks, defenses should also be designed to handle multiple target classes\. Most current defenses can be adapted \(Column 5, Table[II](https://arxiv.org/html/2608.12962#S4.T2)\), but UBD\[[6](https://arxiv.org/html/2608.12962#bib.bib3)\]is incompatible, as it assumes a single target class and finetunes the top model to only reduce the attack effect on that class\.

Fig\. 7:Current defenses are highly sensitive to their hyperparameters\. For instance, the top\-kkclass predictions used for detection in VFLMonitor\[[39](https://arxiv.org/html/2608.12962#bib.bib14)\]can drastically affect its effectiveness yet cannot be tuned in practice\.
#### IV\-B3Hyperparameter Sensitivity

A practical defense should be robust to unseen tasks and should not require hyperparameter tuning, so it should be designed to be insensitive to hyperparameter choices\. However, some defenses cannot meet this requirement \(Column 6, Table[II](https://arxiv.org/html/2608.12962#S4.T2)\)\. For example, VFLMonitor\[[39](https://arxiv.org/html/2608.12962#bib.bib14)\]requires dataset\-specific tuning on itskkvalue for top\-kkpredictions of each party\. As shown in Figure[7](https://arxiv.org/html/2608.12962#S4.F7), this value can significantly affect the trade\-off between detection performance \(Attack TPR\) and false alarm rate \(Clean FPR\) on a VFL dataset\. Gradient noise also suffers from the same problem, as different datasets might have varying training gradient magnitudes and an unmatched noise level might be either ineffective or overly harmful to benign learning\.

### IV\-CPost\-attack Utility Recovery

Current works focus on reducing ASR while overlooking post\-attack utility \(Column 7, Table[II](https://arxiv.org/html/2608.12962#S4.T2)\)\. In fact, even though some defenses claim to recover clean predictions from trigger\-injected ones \(e\.g\., VFLIP\[[8](https://arxiv.org/html/2608.12962#bib.bib4)\], VFLMonitor\[[39](https://arxiv.org/html/2608.12962#bib.bib14)\], and UBD\[[6](https://arxiv.org/html/2608.12962#bib.bib3)\]\), they all fail miserably to preserve recovery utility\. In other words, the attack may be suppressed while predictions remain incorrect\. As widely agreed in general backdoor literature \(e\.g\. Robust Accuracy in BackdoorBench\[[35](https://arxiv.org/html/2608.12962#bib.bib45)\]\), the defense can be considered failed if the recovery attempt cannot restore the correct prediction\.

TABLE III:Current evaluations of VFL backdoor methods are fundamentally flawed\. Prior work fails to compare against state\-of\-the\-art baselines, adopts inconsistent and highly sensitive configurations, relies on unrealistic datasets, and provides fragmented analysis with limited metrics\. The lack of publicly available implementations further undermines reproducibility, making existing results difficult to trust or build upon\.MethodComparisonVFL ConfigurationRealisticDatasetCritical Evaluation AspectsOpenSourceAttackDefenseTop\-DepthBottom\-ArchOptLREfficacyRobustnessDependencyStealthiness\(a\) AttacksBackSplitVFL✩✩✩★✩✩3CNNAdam1e\-3✗★★★★★✩✩✩✩★★✩●LMP★★✩★★★1Tiny\-ResNet?1e\-3✗★★★★✩✩★★✩✩✩✩○PMP★★✩★★★1Tiny\-ResNet?1e\-3✗★★★★✩✩★★✩✩✩✩○VILLAIN★✩✩★★✩3VGG16SGD1e\-2✗★★★★★✩★★★★✩✩◐BadVFL\*★✩✩★✩✩1ResNet18SGD1e\-2✗★★★★✩✩★✩✩✩✩✩●LFBA✩✩✩★✩✩3ResNet18Adam1e\-3✗★★★★✩✩✩✩✩✩✩✩●BadVFL✩✩✩★✩✩4ResNet18??✗★★★★★✩★✩✩★✩✩○HijackVFL✩✩✩★✩✩3ResNet18Adam1e\-3✗★★★★✩✩★✩✩✩✩✩●BAEVFL★★✩★✩✩4ResNet20SGD1e\-1✗★★★★★✩✩✩✩✩✩✩○\(b\) DefensesMethodComparisonVFL ConfigurationRealisticDatasetCritical Evaluation AspectsOpenSourceAttackDefenseTop\-DepthBottom\-ArchOptLREfficacyRobustnessDependencyRecoveryVFLIP★✩✩★★✩3VGG19SGD?✗★★★★✩✩★✩✩✩✩✩◐VFLMonitor★✩✩★✩✩2ResNetAdam1e\-3✗★★★★✩✩★✩✩✩✩✩○UBD★✩✩★✩✩1ResNet18??✗★★★★★✩★★✩✩✩✩○GBD★★✩★★★4ResNet20SGD1e\-1✗★★★★✩✩★★✩✩✩✩●
\[Annotations\]? = not mentioned in the paper or source code\|\|◐= open\-sourced but lack key components

## VBVBench

After identifying the practicality gap in methodological design, a natural question arises:*how do current attacks and defenses perform in practice?*Unfortunately, the answer remains unclear because the current evaluation practices fail to provide a reliable understanding of backdoor vulnerabilities\. A meaningful evaluation should demonstrate improvements over state\-of\-the\-art methods under fair, practical, and comprehensive settings\. However, existing studies fall short along all these dimensions, as summarized in Table[III](https://arxiv.org/html/2608.12962#S4.T3)\.

- •State\-of\-the\-art\.Most works compare against only a few baselines \(Columns 2–3\), partly because reimplementing prior methods requires significant engineering effort, especially when more than half provide no official code \(Column 13\)\. As a result, comparisons are incomplete, making it difficult to assess true progress\.
- •Fairness\.Existing evaluations adopt heterogeneous configurations \(Columns 4–7\), such as different model architectures and hyperparameters\. We find that both attacks and defenses are highly sensitive to these choices\. Even minor changes, such as model depth or optimizer, can significantly affect performance and favor certain methods\. Consequently, reported improvements may reflect configuration bias rather than methodological advances\.
- •Practicality\.Existing studies rely on artificial datasets that do not reflect real\-world VFL deployments \(Column 8\)\. Given the distinct data characteristics of VFL, such simplifications can lead to misleading conclusions\. Our analysis shows that methods performing well on synthetic setups can fail under realistic conditions\.
- •Comprehensiveness\.Evaluations are fragmented, focusing on a limited set of metrics while overlooking other critical aspects \(Columns 9–12\)\. Inconsistent evaluation criteria further hinder meaningful comparison, obscuring the real\-world implications of proposed methods\.

We attribute these shortcomings to the lack of backdoor\-focused benchmarks\. Existing VFL benchmarks primarily emphasize system performance and provide limited support for evaluating backdoor attacks and defenses \(Table[IV](https://arxiv.org/html/2608.12962#S5.T4)\)\. Building such a benchmark is challenging: it requires reimplementing diverse methods within a unified framework while ensuring extensibility for future work\.

TABLE IV:Existing VFL benchmarks are not designed to evaluate backdoor vulnerabilities\. While they support VFL engines and, in some cases, realistic datasets, they provide limited or no support for backdoor vulnerability assessment\.To address these challenges, we introduce BVBench, a backdoor\-focused benchmark designed to enable fair, practical, and comprehensive evaluation of VFL vulnerabilities, preloaded with state\-of\-the\-art attacks and defenses \(Figure[8](https://arxiv.org/html/2608.12962#S5.F8)\)\. Beyond serving as a toolkit, it provides a standardized evaluation framework for future research\. As demonstrated in Section[VI](https://arxiv.org/html/2608.12962#S6)and Section[VII](https://arxiv.org/html/2608.12962#S7), BVBench reveals insights that remain hidden under current evaluation practices\.

![Refer to caption](https://arxiv.org/html/2608.12962v1/benchmark.png)Fig\. 8:BVBench is a modular benchmark for evaluating backdoor vulnerabilities in VFL\. It integrates a unified VFL engine, realistic datasets, and standardized implementations of attacks and defenses into a unified evaluation pipeline, enabling fair, practical, and comprehensive assessment\.### V\-AIngredients

#### V\-A1Unified VFL Engine

We design a unified VFL engine that standardizes model architectures, training configurations, and communication protocols\. The engine strictly enforces the information asymmetry inherent in VFL, ensuring that each party accesses only the information consistent with its role\. It is extensible to different VFL implementations and supports flexible configurations, enabling controlled variation of architectural and optimization choices\.

#### V\-A2Realistic VFL Datasets

We incorporate realistic VFL datasets \(Table[V](https://arxiv.org/html/2608.12962#S5.T5)\) that capture practical feature distributions and cross\-party characteristics\. The benchmark is designed to be easily extensible, allowing new datasets to be integrated with minimal effort\. We also include CIFAR\-10 as a reference for validating reimplementations, since existing methods all adopt this unrealistic dataset for evaluation, providing a common reference point for comparison\.

TABLE V:Commonly used datasets such as CIFAR\-10 \(Row 1\) rely on artificial feature splits, whereas the datasets in BVBench capture natural cross\-party data characteristics\.
#### V\-A3State\-of\-the\-art Attacks and Defenses

We curate a comprehensive collection of state\-of\-the\-art attacks and defenses from major venues\. The initial set includes nine attacks and eight defenses, all implemented in PyTorch with over 20,000 lines of code\. While some methods provide open\-source implementations and can be easily adapted to BVBench, others require reimplementation from scratch\. For methods whose reproduced performance does not match the originally reported results, we have contacted the authors for clarification and assistance\. The framework is designed for extensibility, allowing new methods to be integrated with minimal code changes\.

### V\-BEvaluation Recipes

BVBench provides standardized evaluation recipes for attacks and defenses\. The recipes define the evaluation aspects and associated metrics that should be reported to enable fair, practical, and reproducible comparison\. Detailed formulations are deferred to the appendix\.

#### V\-B1Attack Evaluation

A high attack success rate alone is insufficient to establish a practical attack\. In realistic deployments, attacks must also preserve model utility, operate under limited prior knowledge, remain stealthy, generalize across settings, and avoid excessive overhead\. We therefore evaluate attacks along the following dimensions\.

- •Efficacy:Evaluate attacks using both main task accuracy \(MTA\) and attack success rate \(ASR\) across diverse datasets\. Since aggregate metrics can mask heterogeneity, conduct per\-class analysis to examine variations across target classes and multi\-target analysis to assess effectiveness when multiple classes are attacked simultaneously\.
- •Dependency:Quantify how attack effectiveness changes under different levels of required prior knowledge, such as imperfect label knowledge, noisy label inference, or other attack\-specific prerequisites, and measure their impact on both ASR and MTA\.
- •Stability:Assess the consistency of attack performance across random seeds and training epochs, rather than reporting only the final checkpoint, using statistics such as the mean and variance of MTA and ASR\.
- •Robustness:Examine attack performance under different VFL environments, including varying numbers of participating parties, and multiple attackers with either shared or conflicting objectives\.
- •Stealthiness:Assess the distinguishability between poisoned and benign embeddings\. We introduce stealthiness metrics that capture both abnormal embedding magnitudes and deviations from benign embedding distributions\. We consider both an oracle setting based on class\-conditional distributions and a label\-free setting based on global embedding distributions, and quantify the separability between benign and malicious samples using ROC\-AUC\.
- •Overhead:Report the computational, communication, memory, and optimization costs incurred by the attack\.

#### V\-B2Defense Evaluation

Reducing attack success alone is insufficient to establish a practical defense\. Defenses must preserve normal utility, remain robust across attacks and datasets, and avoid excessive cost\. We therefore evaluate defenses along the following dimensions\.

- •Efficacy:Evaluate defenses using MTA, defended ASR, and utility recovery \(UR\) across diverse datasets and attacks\. Per\-class and multi\-target analyses should further assess whether protection remains balanced across target classes and attack objectives\.
- •Dependency:Assess how defense effectiveness depends on required assumptions or prerequisites, such as clean validation data, trusted reference samples, prior attack knowledge, or defense\-specific hyperparameter tuning\.
- •Stability:Measure the variability of MTA, defended ASR, and UR across random seeds and repeated runs to ensure that reported protection is reproducible\.
- •Robustness:Evaluate defenses under diverse adversarial environments, including different numbers of parties and scenarios involving multiple attackers with shared or conflicting targets\.
- •Overhead:Report the computation, communication, memory, and inference costs introduced by the defense\.

Together, these evaluation recipes provide a standardized and practice\-oriented framework for assessing VFL backdoor attacks and defenses\.

TABLE VI:BVBench compares attacks on the same VFL setting across realistic datasets for fairness and generating practical insights\. Current attacks can compromise main task accuracy, and their attack success is not as eye\-catching as reported\.

## VIEmpirical Analysis: Attacks

We empirically analyze existing attacks using BVBench\. To the best of our knowledge, this is the first study to compare VFL backdoor attacks under a unified benchmark that enforces fairness, practicality, and comprehensive evaluation\. Due to the space limit, we highlight key findings here, which reveal that many conclusions drawn from prior studies fail to generalize once these factors are controlled\.

### VI\-AEfficacy

The conclusions derived from CIFAR\-10, the unrealistic dataset commonly used in prior work, differ substantially from those obtained on realistic VFL datasets\. As a prerequisite for qualifying as effective attacks, Table[VI](https://arxiv.org/html/2608.12962#S5.T6)reports the class\-averaged MTA and ASR of all methods across datasets\. Although all attacks preserve MTA on CIFAR\-10, they can incur substantial utility degradation on realistic datasets, reducing MTA by as much as 13\.39% on Satellite, 50\.50% on KUHAR, and 17\.78% on PTB\-XL\. The findings are even more concerning for ASR\. First, despite prior studies often reporting ASR exceeding 80% on CIFAR\-10, BadVFL, LFBA, and VILLAIN achieve less than 30% ASR under our evaluation, while most remaining methods fail to exceed 70%\. This discrepancy stems from the fact that BVBench enforces configuration fairness: all methods are evaluated under the same VFL environment rather than under method\-specific settings\. Second, no existing attack demonstrates consistently strong performance across datasets\. The gap between the best and worst cases can be substantial, with ASR varying by as much as 92\.39% for BadVFL∗\. Collectively, these results suggest that current VFL backdoor attacks are considerably less effective than previously implied\. Robust attack efficacy across diverse VFL environments remains largely an open problem\.

Per\-Class Analysis\.Existing studies typically report ASR for a selected target class, obscuring variations in attack effectiveness across classes\. To expose this phenomenon, we repeat each attack on Satellite using different target classes\. Figure[9](https://arxiv.org/html/2608.12962#S6.F9)\(color bars\) presents the resulting per\-class ASR\. Almost all attacks achieve their highest ASR when targeting Class 2 \(green\), which corresponds to the majority class\. This observation suggests that attack success is strongly influenced by class distribution and that reported performance can be sensitive to target selection\.

Fig\. 9:Existing attacks achieve high ASR primarily when targeting the majority class \(Class 2\), and this bias persists under both single\-target and multi\-target settings\.Multi\-Target Analysis\.This tendency becomes even more pronounced in the multi\-target setting \(hollow bars in Figure[9](https://arxiv.org/html/2608.12962#S6.F9)\)\. Because multi\-target learning is inherently more challenging, attacks appear to preferentially optimize shortcuts associated with majority classes when trade\-offs arise\. Consequently, ASR on minority classes can deteriorate drastically\. For example, BackSplitVFL achieves 100% ASR on the majority class while completely failing on all others\. Since multi\-target capability is fundamental to the practical backdoor workflow \(Section[III\-B](https://arxiv.org/html/2608.12962#S3.SS2)\), improving the balance of effectiveness across targets represents an important direction for future research\.

### VI\-BDependency

Backdoor attacks rely on label knowledge, and the quality of such knowledge can critically influence attack success\. Using LFBA, the best\-performing attack in our benchmark and one that does not assume perfectly labeled samples as prior knowledge, as a case study, Figure[10](https://arxiv.org/html/2608.12962#S6.F10)reports its ASR under different levels of manually controlled label quality \(solid lines\)\.

Fig\. 10:LFBA’s attack effectiveness is highly sensitive to label knowledge quality \(solid lines\), while its label inference component exhibits significant variability across datasets \(dashed lines\)\.Three observations emerge\. First, perfect label knowledge does not necessarily guarantee successful attacks\. Even when true labels are provided, LFBA still fails completely on Satellite\. Second, on datasets where LFBA is effective \(e\.g\., Vehicle\), label quality becomes decisive: reducing label quality by only 20% can decrease ASR by more than 25%\. Third, the label inference procedure itself exhibits strong dataset dependence\. It performs well on some datasets \(e\.g\., Vehicle\) but fails on others \(e\.g\., Satellite and PTB\-XL\), as indicated by the dashed lines\. These findings suggest that label knowledge is neither sufficient nor uniformly obtainable in practice\. While improving label inference remains important, future work should also investigate why attacks remain ineffective even when accurate label information is available\.

### VI\-CStability Under Randomness

Effective attacks should exhibit consistent performance across repeated executions\. However, existing methods can be highly sensitive to randomness\. An attack may perform well during one epoch but fail in the next, or succeed under one random seed while collapsing under another\. Figure[11](https://arxiv.org/html/2608.12962#S6.F11)illustrates a representative example using BAEVFL on Vehicle\. Its ASR drops from 100% to 0% within two consecutive epochs and recovers to 100% only two epochs later\. Changing the random seed introduces further variability\. Such instability raises concerns regarding the reliability of current evaluations\. An attack may appear successful simply because training happens to terminate at a favorable point along the optimization trajectory\. Therefore, future evaluations should systematically assess sensitivity to randomness and demonstrate that attack effectiveness remains stable across both epochs and random seeds\.

Fig\. 11:BAEVFL’s ASR on Vehicle fluctuates substantially across epochs and is highly sensitive to randomness, highlighting the importance of stability and reproducibility for meaningful attack evaluation\.
### VI\-DRobustness to Adversarial Environment

Backdoor attacks should remain effective regardless of how many peers participate in the VFL system and whether those peers behave benignly or maliciously\.

The number of peers affects the fraction of top\-model inputs controlled by the adversary\. Using Satellite as a case study, Figure[12](https://arxiv.org/html/2608.12962#S6.F12)shows that ASR is strongly influenced by the number of participating parties\. Embedding\-space attacks \(solid lines\) consistently outperform input\-space attacks \(dashed lines\), regardless of whether the attacker controls 50% of the top\-model inputs \(two parties\) or only 6\.25% \(sixteen parties\)\. A plausible explanation is that input\-space triggers must propagate through the bottom model before influencing the top model, weakening their effect, whereas embedding\-space attacks directly manipulate the exchanged representations and bypass this attenuation\.

Fig\. 12:Embedding\-space attacks \(solid lines\) tend to perform better than input\-space attacks \(dashed lines\) on Satellite, no matter the attacker controls more or less features to the top model\.Another overlooked aspect in the adversarial environment concerns the presence of multiple malicious parties\. Using NUSWIDE as a case study, Figure[13](https://arxiv.org/html/2608.12962#S6.F13)compares the single\-attacker setting \(solid lines\) with two\-attacker settings involving either shared targets \(green bars\) or conflicting targets \(red bars\)\. Across attacks, coordinated attackers targeting the same class substantially improve ASR relative to the single\-attacker baseline\. In contrast, independent attackers pursuing different targets interfere with one another and significantly degrade attack effectiveness\.

Fig\. 13:Compared with the single\-attacker baseline \(solid lines\), multiple colluding attackers \(green\) with consistent targets can reinforce attacks, but independent attackers with conflicting targets \(red\) can interfere with each other\.These observations suggest that current evaluations underestimate the role of interactions among participants\. Understanding how attacks behave under realistic mixtures of benign and adversarial peers remains an important open question\.

### VI\-EStealthiness

Train\-Time Stealthiness\.Trigger\-injected samples should remain indistinguishable from benign samples throughout training and inference\. Although BackSplitVFL and VILLAIN explicitly constrain embeddings to improve stealthiness, the resulting malicious embeddings remain readily distinguishable under the stealthiness score introduced in Section[V](https://arxiv.org/html/2608.12962#S5)\.

Fig\. 14:Distribution of stealthiness for clean samples and attack samples\. Most clean samples stay in a high level while attacked samples are at a significantly low stealthiness level\.Figure[14](https://arxiv.org/html/2608.12962#S6.F14)illustrates the distribution of stealthiness scores for benign and malicious embeddings generated by BackSplitVFL\. The resulting bipolar distribution clearly exposes the presence of poisoning, while the consistently low scores of malicious embeddings can even reveal which samples are poisoned\. Consequently, the active party may identify malicious participants during training\. This separability is positively correlated with attack effectiveness\. Figure[15](https://arxiv.org/html/2608.12962#S6.F15)reports the ROC\-AUC between malicious and benign samples\. When ASR is high, the ROC\-AUC of stealthiness metrics approaches 1\.0, indicating near\-perfect detectability\. In contrast, when attacks are largely ineffective \(ASR<<20%\), the ROC\-AUC drops to approximately 0\.6–0\.7\. Under our simple and intuitive stealthiness metrics, current attacks therefore fail to achieve a favorable balance between efficacy and stealthiness\.

Fig\. 15:When an attack is effective, the ROC\-AUC can be close to 1\.0, suggesting a strong separation between clean samples and attack samples\. However, when an attack fails, the ROC\-AUC can drop to 0\.6\-0\.7\.Test\-Time Stealthiness\.Stealthiness scores can also be leveraged during deployment, where the active party lacks ground\-truth labels but seeks to identify suspicious queries\. As shown in Figure[16](https://arxiv.org/html/2608.12962#S6.F16), in this label\-free setting, we observe strong agreement with the oracle setting that requires labels and is more suitable for training\-time detection\. In particular, the correlation between label\-free and oracle ROC\-AUC can reach 0\.98\. This finding suggests that label\-free stealthiness estimation can serve as a practical approximation for both attackers and defenders\.

Fig\. 16:The oracle mode and the label\-free mode of our stealthiness metric can both offer strong detectability for effective attacks\.Overall, our results indicate that current VFL backdoor attacks are not stealthy from the defender’s perspective\. Effective attacks tend to be highly anomalous, whereas stealthier attacks are often ineffective\. Whether future methods can achieve a substantially better efficacy\-stealthiness trade\-off remains an open question\. Otherwise, practical VFL backdoor attacks may be considerably easier to detect than previously assumed\.

## VIIEmpirical Analysis: Defenses

By systematically comparing VFL backdoor defenses under a unified benchmark, BVBench also reveals previously overlooked limitations of existing methods\. We summarize the key findings below\.

TABLE VII:No existing defense can simultaneously preserve benign utility, suppress attacks, and recover correct predictions on attacked samples\. ASR, MTA, and UR are averaged over all attacks and target classes\.### VII\-AEfficacy

Current defenses fail to simultaneously preserve utility, suppress attacks, and recover correct predictions\. Defense efficacy should be evaluated along three dimensions: maintaining main task accuracy \(MTA\) in the absence of attacks, reducing the attack success rate \(ASR\) of trigger\-injected queries, and achieving utility recovery \(UR\), i\.e\., correctly classifying those trigger\-injected queries after mitigation\. Table[VII](https://arxiv.org/html/2608.12962#S7.T7)summarizes the performance of all defenses across datasets, averaged over all attacks and target classes\. Three observations emerge\. First, existing defenses exhibit a serious utility\-security trade\-off\. Methods that effectively suppress attacks often incur substantial degradation in clean performance, whereas methods that preserve utility typically provide only limited protection\. For example, VFLIP achieves the largest ASR reduction on KUHAR, lowering ASR from 20\.48% to 6\.18%, but reduces MTA to only 0\.31%\. In contrast, GBD best preserves utility, with MTA decreasing only from 62\.81% to 59\.98%, yet its defended ASR remains at 15\.23%\. Similar trade\-offs are observed across other defenses and datasets\. Second, no existing defense consistently achieves strong performance across datasets\. Similar to attacks, defense effectiveness is highly dataset\-dependent\. A method that performs well in one setting may fail completely in another, suggesting that current defenses do not generalize reliably across realistic VFL environments\. Third, existing defenses struggle to recover correct predictions after mitigating attacks\. Ideally, UR should approach MTA, indicating that the defense not only suppresses targeted misclassification but also restores the model’s intended functionality\. However, Table[VII](https://arxiv.org/html/2608.12962#S7.T7)shows that no defense achieves this objective, including methods explicitly designed for recovery\. For example, although VFLMonitor reduces ASR from 73\.68% to 29\.03% on Vehicle, its UR reaches only 31\.79%, far below its MTA of 62\.52%\. Similarly, VFLIP lowers ASR to 33\.62%, yet its UR remains substantially lower than MTA \(21\.30% versus 83\.70%\)\. These results indicate that disrupting targeted attacks alone is insufficient, as defended samples often remain misclassified\. A plausible explanation is the limited reliability of the anomaly detection mechanisms underlying repair\-based defenses\. Figure[17](https://arxiv.org/html/2608.12962#S7.F17)examines the detection performance of representative methods\. VFLMonitor fails to simultaneously achieve a high true positive rate \(TPR\) and a low false positive rate \(FPR\), while VFLIP exhibits strong detection capability on some datasets \(e\.g\., Vehicle\) but poor generalization to others \(e\.g\., PTB\-XL\)\. Overall, future defenses should place greater emphasis on utility recovery, as preserving prediction correctness is ultimately more important than merely reducing ASR\.

Fig\. 17:Existing detection\-based defenses fail to reliably distinguish attacked samples, preventing them from achieving both high TPR and low FPR across datasets\.Per\-Class Analysis\.Existing studies report class\-averaged metrics, which obscure variations in protection across target classes\. To expose this phenomenon, Figure[18](https://arxiv.org/html/2608.12962#S7.F18)reports defended ASR for different target classes on the same dataset\. Compared with the undefended baseline \(solid lines\), most defenses fail to consistently suppress attacks across classes\. Interestingly, VFLIP reduces the ASR of Classes 0, 1, and 3 to below 10%, yet strengthens the attack on the majority class \(Class 2\)\. Similar trends are observed for gradient noise and embedding normalization\. These results suggest that current defenses provide uneven protection and may leave certain classes substantially more vulnerable than others\.

Fig\. 18:Most defenses fail to provide consistent protection across target classes on Satellite\. ASR under defense may even surpass the scenario with no defense \(solid lines\)\.Multi\-Target Analysis\.The limitations of current defenses become even more pronounced in the multi\-target setting\. Figure[19](https://arxiv.org/html/2608.12962#S7.F19)shows that the defended ASR of most methods remains close to the single\-target baseline\. Moreover, the influence of class imbalance becomes stronger, with defenses becoming less effective at protecting majority classes from attack\. Most defenses exhibit similar robustness under both single\-target and multi\-target settings despite the latter posing a substantially stronger threat\. Since existing methods lack mechanisms for balancing defense effectiveness across classes or attack objectives, improving robustness against multi\-target attacks remains an important challenge for future research\.

Fig\. 19:Multi\-target attacks further expose the limitations of current defenses: most methods fail to reduce ASR across classes, and the majority\-class effect becomes even more pronounced\.
### VII\-BRobustness to Adversarial Environment

Current defenses are fragile under realistic multi\-adversary environments\. Figure[20](https://arxiv.org/html/2608.12962#S7.F20)reports defended ASR \(bars\) and undefended ASR \(solid lines\) under two scenarios: \(i\) multiple attackers sharing the same target and \(ii\) multiple attackers pursuing conflicting targets\. We report the ASRs of one of the attackers\. Two observations emerge\. First, existing defenses provide only limited protection against colluding attackers\. Across methods, the defended ASR under shared\-target attacks \(green bars\) remains close to the undefended baseline, indicating that current defenses struggle to mitigate coordinated adversarial behavior\. Second, defenses can inadvertently strengthen attacks by redistributing protection unevenly across targets\. For example, VFLIP may successfully suppress one adversary’s backdoor while simultaneously enabling another adversary to achieve higher attack success\. This phenomenon is particularly concerning because it remains hidden when evaluation focuses solely on aggregate metrics\. These findings suggest that current defenses are not robust to complex adversarial environments involving multiple malicious participants\. Future defense designs should explicitly account for interactions among attackers and ensure balanced protection across target classes and attack objectives\.

Fig\. 20:Current defenses are not robust to multi\-adversary settings\. Compared with the ASR under no defense \(solid lines\), they provide only marginal protection against colluding attackers and can even inadvertently facilitate attacks when adversaries target different classes\.

## VIIIDiscussions

By systematizing existing work and completing overlooked practicality dimensions through BVBench, we show that current understanding of VFL backdoor vulnerability requires substantial revision\. Under realistic threat models, standardized evaluation, and comprehensive criteria, most VFL backdoor attacks and defenses are far less practical than previously reported\.

On the attack side, no existing attack remains practical, because they all rely on inaccessible task\-related knowledge\. Even with such knowledge, their effectiveness is still limited by strong dependency, instability, multi\-target incompatibility, poor stealthiness, and sensitivity to the VFL environment\. Current attacks are hard to justify under realistic assumptions and difficult to sustain in practice\.

This does not mean VFL systems are already well protected against vulnerability\. Current defenses rely on unreasonable knowledge and fail to consistently achieve all three defense objectives\. These limitations worsen in multi\-target and multi\-adversary settings, where protection can be uneven across classes and fragile to the attack composition\.

Our findings carry useful insights for different stakeholders\. Future work on attacks and defenses should follow the practical threat model and workflow while meeting practical requirements, such as attack stealthiness and defense utility recovery\. Practitioners who want to deploy backdoor\-resilient VFL systems should rely on fair and comprehensive evaluation results under BVBench rather than heterogeneous reported numbers\.

Our study is not exhaustive\. We focus on neural\-network\-based VFL and single\-label classification, while real\-world VFL may involve other tasks or protocols\[[36](https://arxiv.org/html/2608.12962#bib.bib25),[25](https://arxiv.org/html/2608.12962#bib.bib39),[4](https://arxiv.org/html/2608.12962#bib.bib24),[32](https://arxiv.org/html/2608.12962#bib.bib40)\]\. Besides, this work aims to recalibrate the understanding of practical VFL backdoor vulnerabilities rather than propose a new practical attack or defense\. We hope BVBench can serve as a useful tool to steer future VFL backdoor research toward a more practical direction\.

## IXConclusions

Current VFL backdoor research has made significant progress, but its practicality has not been fairly and fully evaluated\. We find that the practicality gaps lie not only in methodology designs, but also in evaluation designs\. Our findings through systematization and empirical analysis from BVBench suggest that current understanding of VFL backdoor vulnerability is incomplete and unreliable, and should be calibrated under the practical threat model and evaluation recipe to guide future research\.

## References

- \[1\]\(2023\)ADI: Adversarial dominating inputs in vertical federated learning systems\.In2023 IEEE Symposium on Security and Privacy \(SP\),pp\. 1875–1892\.External Links:[Document](https://dx.doi.org/10.1109/SP46215.2023.10179446),ISBN 978\-1\-6654\-9336\-9Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1)\.
- \[2\]Y\. Bai, Y\. Chen, H\. Zhang, W\. Xu, H\. Weng, and D\. Goodman\(2023\)VILLAIN: Backdoor Attacks Against Vertical Split Learning\.In32nd USENIX Security Symposium \(USENIX Security 23\),pp\. 2743–2760\.External Links:ISBN 978\-1\-939133\-37\-3Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p4.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.7.1),[1st item](https://arxiv.org/html/2608.12962#S3.I1.i1.p1.1),[§III\-B2](https://arxiv.org/html/2608.12962#S3.SS2.SSS2.p1.1),[§III\-C](https://arxiv.org/html/2608.12962#S3.SS3.p1.1)\.
- \[3\]California Legislature\(2018\)California consumer privacy act of 2018, Cal\. Civ\. Code § 1798\.100 et seq\.\.Note:Amended by the California Privacy Rights Act \(CPRA\) of 2020Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p1.1)\.
- \[4\]C\. Chen, J\. Zhou, L\. Zheng, H\. Wu, L\. Lyu, J\. Wu, B\. Wu, Z\. Liu, L\. Wang, and X\. Zheng\(2022\)Vertically federated graph neural network for privacy\-preserving node classification\.InProceedings of the Thirty\-First International Joint Conference on Artificial Intelligence, IJCAI\-22,L\. D\. Raedt \(Ed\.\),pp\. 1959–1965\.Note:Main TrackExternal Links:[Document](https://dx.doi.org/10.24963/ijcai.2022/272)Cited by:[§VIII](https://arxiv.org/html/2608.12962#S8.p5.1)\.
- \[5\]J\. Chen, D\. Chen, J\. Cui, and H\. Zhong\(2025\)Backdoor Attack on Encryption\-Protected Vertical Federated Learning\.IEEE Transactions on Information Forensics and Security20,pp\. 6968–6983\.External Links:ISSN 1556\-6013, 1556\-6021,[Document](https://dx.doi.org/10.1109/TIFS.2025.3581095)Cited by:[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.14.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.12.1)\.
- \[6\]P\. Chen, H\. Xiang, X\. Du, X\. Xu, X\. Jiang, Z\. Lu, J\. Yang, Q\. Duan, and W\. Dou\(2025\)Universal Backdoor Defense via Label Consistency in Vertical Federated Learning\.InProceedings of the Thirty\-Fourth International Joint Conference on Artificial Intelligence,Vol\.1,pp\. 4743–4751\.External Links:ISSN 1045\-0823,[Document](https://dx.doi.org/10.24963/ijcai.2025/528)Cited by:[§IV\-B2](https://arxiv.org/html/2608.12962#S4.SS2.SSS2.p1.1),[§IV\-C](https://arxiv.org/html/2608.12962#S4.SS3.p1.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.7.1)\.
- \[7\]X\. Chen, C\. Liu, B\. Li, K\. Lu, and D\. Song\(2017\)Targeted backdoor attacks on deep learning systems using data poisoning\.InarXiv preprint arXiv:1712\.05526,Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p1.1)\.
- \[8\]Y\. Cho, W\. Han, M\. Yu, Y\. Lee, H\. Bae, and Y\. Paek\(2024\)VFLIP: A Backdoor Defense for Vertical Federated Learning via Identification and Purification\.InEuropean Symposium on Research in Computer Security,J\. Garcia\-Alfaro, R\. Kozik, M\. Choraś, and S\. Katsikas \(Eds\.\),Vol\.14985,Cham,pp\. 291–312\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-70903-6%5F15),ISBN 978\-3\-031\-70903\-6Cited by:[§IV\-C](https://arxiv.org/html/2608.12962#S4.SS3.p1.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.5.1)\.
- \[9\]European Parliament and Council of the European Union\(2016\)Regulation \(EU\) 2016/679 of the European Parliament and of the Council \(GDPR\)\.External Links:[Link](https://data.europa.eu/eli/reg/2016/679/oj)Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p1.1)\.
- \[10\]C\. Fu, X\. Zhang, S\. Ji, J\. Chen, J\. Wu, S\. Guo, J\. Zhou, A\. X\. Liu, and T\. Wang\(2022\)Label Inference Attacks Against Vertical Federated Learning\.In31st USENIX Security Symposium \(USENIX Security 22\),pp\. 1397–1414\.External Links:ISBN 978\-1\-939133\-31\-1Cited by:[2nd item](https://arxiv.org/html/2608.12962#S2.I2.i2.p1.1)\.
- \[11\]C\. Fung, C\. J\. M\. Yoon, and I\. Beschastnikh\(2020\)The limitations of federated learning in sybil settings\.InProceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses,pp\. 301–316\.Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p2.1)\.
- \[12\]D\. Gao, S\. Wan, H\. Gu, L\. Fan, X\. Yao, and Q\. Yang\(2024\)Label Privacy Source Coding in Vertical Federated Learning\.InMachine Learning and Knowledge Discovery in Databases\. Research Track,A\. Bifet, J\. Davis, T\. Krilavičius, M\. Kull, E\. Ntoutsi, and I\. Žliobaitė \(Eds\.\),Vol\.14941,Cham,pp\. 313–331\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-70341-6%5F19),ISBN 978\-3\-031\-70341\-6Cited by:[2nd item](https://arxiv.org/html/2608.12962#S2.I2.i2.p1.1)\.
- \[13\]T\. Gu, B\. Dolan\-Gavitt, and S\. Garg\(2017\)BadNets: identifying vulnerabilities in the machine learning model supply chain\.arXiv preprint arXiv:1708\.06733\.Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p1.1)\.
- \[14\]Y\. He, Z\. Shen, J\. Hua, Q\. Dong, J\. Niu, W\. Tong, X\. Huang, C\. Li, and S\. Zhong\(2024\)Backdoor Attack Against Split Neural Network\-Based Vertical Federated Learning\.IEEE Transactions on Information Forensics and Security19,pp\. 748–763\.External Links:ISSN 1556\-6013, 1556\-6021,[Document](https://dx.doi.org/10.1109/TIFS.2023.3327853)Cited by:[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.4.1),[Fig\. 6](https://arxiv.org/html/2608.12962#S3.F6),[Fig\. 6](https://arxiv.org/html/2608.12962#S3.F6.4),[1st item](https://arxiv.org/html/2608.12962#S3.I1.i1.p1.1),[§III\-C](https://arxiv.org/html/2608.12962#S3.SS3.p1.1)\.
- \[15\]C\. Huang, L\. Wang, and X\. Han\(2023\)Vertical federated knowledge transfer via representation distillation for healthcare collaboration networks\.InProceedings of the ACM Web Conference 2023,pp\. 4188–4199\.Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p1.1)\.
- \[16\]A\. Khan, M\. ten Thij, and A\. Wilbik\(2025\)Vertical federated learning: A structured literature review\.67\(4\),pp\. 3205–3243\.External Links:ISSN 0219\-3116,[Document](https://dx.doi.org/10.1007/s10115-025-02356-y)Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p2.1)\.
- \[17\]A\. Krizhevsky G\. Hintonet al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.2.1)\.
- \[18\]J\. Liu, X\. Lyu, C\. Ren, and Q\. Cui\(2025\)Targeted Poisoning Attacks Against Vertical Federated Learning via Embedding Manipulation\.IEEE Transactions on Dependable and Secure Computing22\(6\),pp\. 7535–7551\.External Links:ISSN 1545\-5971, 1941\-0018, 2160\-9209,[Document](https://dx.doi.org/10.1109/TDSC.2025.3598683)Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p4.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.5.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.6.1),[1st item](https://arxiv.org/html/2608.12962#S3.I1.i1.p1.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.11.1)\.
- \[19\]K\. Liu, B\. Dolan\-Gavitt, and S\. Garg\(2018\)Fine\-pruning: defending against backdooring attacks on deep neural networks\.InResearch in Attacks, Intrusions, and Defenses \(RAID\),pp\. 273–294\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-00470-5%5F13)Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p2.1)\.
- \[20\]Y\. Liu, Y\. Kang, T\. Zou, Y\. Pu, Y\. He, X\. Ye, Y\. Ouyang, Y\. Zhang, and Q\. Yang\(2024\)Vertical Federated Learning: Concepts, Advances, and Challenges\.IEEE Transactions on Knowledge and Data Engineering36\(7\),pp\. 3615–3634\.External Links:ISSN 1041\-4347, 1558\-2191, 2326\-3865,[Document](https://dx.doi.org/10.1109/TKDE.2024.3352628)Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p2.1)\.
- \[21\]Y\. Liu, Y\. Lou, Y\. Liu, Y\. Cao, and H\. Wang\(2024\)Label leakage in vertical federated learning: A survey\.InThirty\-Third International Joint Conference on Artificial Intelligence,Vol\.9,pp\. 8160–8169\.External Links:ISSN 1045\-0823,[Document](https://dx.doi.org/10.24963/ijcai.2024/902)Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1)\.
- \[22\]M\. Naseri, Y\. Han, and E\. De Cristofaro\(2024\)BadVFL: Backdoor Attacks in Vertical Federated Learning\.In2024 IEEE Symposium on Security and Privacy \(SP\),pp\. 2013–2028\.External Links:[Document](https://dx.doi.org/10.1109/SP54263.2024.00008)Cited by:[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.10.1),[2nd item](https://arxiv.org/html/2608.12962#S3.I1.i2.p1.1)\.
- \[23\]T\. D\. Nguyen, P\. Rieger, H\. Chen, H\. Yalame, H\. Möllering, H\. Fereidooni, S\. Marchal, M\. Miettinen, A\. Mirhoseini, S\. Zeitouni, F\. Koushanfar, A\. Sadeghi, and T\. Schneider\(2022\)FLAME: taming backdoors in federated learning\.In31st USENIX Security Symposium \(USENIX Security 22\),pp\. 1415–1432\.Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p2.1)\.
- \[24\]B\. Pinkas, T\. Schneider, and M\. Zohner\(2018\)Scalable private set intersection based on OT extension\.InACM Transactions on Privacy and Security \(TOPS\),Vol\.21,pp\. 1–35\.Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p3.1)\.
- \[25\]B\. Qian, Y\. Xie, Y\. Li, B\. Ding, and J\. Zhou\(2025\)Tree\-based models for vertical federated learning: A survey\.57\(9\),pp\. 241:1–241:30\.External Links:ISSN 0360\-0300,[Document](https://dx.doi.org/10.1145/3728314)Cited by:[§VIII](https://arxiv.org/html/2608.12962#S8.p5.1)\.
- \[26\]P\. Qiu, X\. Zhang, S\. Ji, C\. Li, Y\. Pu, X\. Yang, and T\. Wang\(2024\)Hijack Vertical Federated Learning Models As One Party\.IEEE Transactions on Dependable and Secure Computing,pp\. 1–18\.External Links:ISSN 1545\-5971, 1941\-0018, 2160\-9209,[Document](https://dx.doi.org/10.1109/TDSC.2024.3358081)Cited by:[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.11.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.12.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.13.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.9.1)\.
- \[27\]W\. Shen, W\. Huang, G\. Wan, and M\. Ye\(2025\)Label\-Free Backdoor Attacks in Vertical Federated Learning\.Proceedings of the AAAI Conference on Artificial Intelligence39\(19\),pp\. 20389–20397\.External Links:ISSN 2374\-3468, 2159\-5399,[Document](https://dx.doi.org/10.1609/aaai.v39i19.34246)Cited by:[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.9.1),[§III\-B2](https://arxiv.org/html/2608.12962#S3.SS2.SSS2.p1.1)\.
- \[28\]W\. Shen, W\. Liu, M\. Chen, W\. Huang, and M\. Ye\(2025\)MARS\-VFL: A unified benchmark for vertical federated learning with realistic evaluation\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1),[TABLE IV](https://arxiv.org/html/2608.12962#S5.T4.5.4.1),[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.4.1),[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.5.1)\.
- \[29\]B\. Tran, J\. Li, and A\. Madry\(2018\)Spectral signatures in backdoor attacks\.InAdvances in Neural Information Processing Systems,Vol\.31\.Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p2.1)\.
- \[30\]P\. Vepakomma, O\. Gupta, T\. Swedish, and R\. Raskar\(2018\)Split learning for health: distributed deep learning without sharing raw patient data\.arXiv preprint arXiv:1812\.00564\.Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p1.1),[§I](https://arxiv.org/html/2608.12962#S1.p2.1)\.
- \[31\]B\. Wang, Y\. Yao, S\. Shan, H\. Li, B\. Viswanath, H\. Zheng, and B\. Y\. Zhao\(2019\)Neural cleanse: identifying and mitigating backdoor attacks in neural networks\.In2019 IEEE Symposium on Security and Privacy \(SP\),pp\. 707–723\.External Links:[Document](https://dx.doi.org/10.1109/SP.2019.00031)Cited by:[§II\-B](https://arxiv.org/html/2608.12962#S2.SS2.p2.1)\.
- \[32\]G\. Wang, Q\. Li, X\. Liu, X\. Yan, Q\. Dong, H\. Wu, X\. Kong, and L\. Zhou\(2025\)VFGCN: A vertical federated learning framework with privacy preserving for graph convolutional network\.22\(5\),pp\. 5617–5631\.External Links:ISSN 1941\-0018,[Document](https://dx.doi.org/10.1109/TDSC.2025.3570626)Cited by:[§VIII](https://arxiv.org/html/2608.12962#S8.p5.1)\.
- \[33\]G\. Wang, B\. Gu, Q\. Zhang, X\. Li, B\. Wang, and C\. X\. Ling\(2024\)A unified solution for privacy and communication efficiency in vertical federated learning\.Advances in Neural Information Processing Systems36\.Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1)\.
- \[34\]P\. Wei, H\. Dou, S\. Liu, R\. Tang, L\. Liu, L\. Wang, and B\. Zheng\(2023\)FedAds: A benchmark for privacy\-preserving CVR estimation with vertical federated learning\.InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR ’23,pp\. 3037–3046\.External Links:[Document](https://dx.doi.org/10.1145/3539618.3591909),ISBN 978\-1\-4503\-9408\-6Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1)\.
- \[35\]B\. Wu, H\. Chen, M\. Zhang, Z\. Zhu, S\. Wei, D\. Yuan, and C\. Shen\(2022\)BackdoorBench: a comprehensive benchmark of backdoor learning\.InThirty\-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§IV\-C](https://arxiv.org/html/2608.12962#S4.SS3.p1.1)\.
- \[36\]Y\. Wu, S\. Cai, X\. Xiao, G\. Chen, and B\. C\. Ooi\(2020\)Privacy preserving vertical federated learning for tree\-based models\.Proceedings of the VLDB Endowment13\(12\),pp\. 2090–2103\.Cited by:[§VIII](https://arxiv.org/html/2608.12962#S8.p5.1)\.
- \[37\]Z\. Wu, J\. Hou, and B\. He\(2024\)VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks\.InThe Twelfth International Conference on Learning Representations,Cited by:[TABLE IV](https://arxiv.org/html/2608.12962#S5.T4.5.3.1),[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.3.1),[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.6.1),[TABLE V](https://arxiv.org/html/2608.12962#S5.T5.5.7.1)\.
- \[38\]Y\. Xi, Y\. Guo, S\. Xu, C\. Cai, and X\. Jia\(2025\)Private sample alignment for vertical federated learning: An efficient and reliable realization\.20,pp\. 3834–3848\.External Links:ISSN 1556\-6021,[Document](https://dx.doi.org/10.1109/TIFS.2025.3555794)Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p3.1)\.
- \[39\]X\. Xu, Y\. Zhao, Y\. Han, Y\. Zhu, Z\. Han, G\. Xu, B\. Wang, S\. Ji, and W\. Wang\(2025\)VFLMonitor: Defending One\-Party Hijacking Attacks in Vertical Federated Learning\.IEEE Transactions on Information Forensics and Security20,pp\. 4828–4843\.External Links:ISSN 1556\-6013, 1556\-6021,[Document](https://dx.doi.org/10.1109/TIFS.2025.3564879)Cited by:[Fig\. 7](https://arxiv.org/html/2608.12962#S4.F7),[Fig\. 7](https://arxiv.org/html/2608.12962#S4.F7.4),[§IV\-B3](https://arxiv.org/html/2608.12962#S4.SS2.SSS3.p1.1),[§IV\-C](https://arxiv.org/html/2608.12962#S4.SS3.p1.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.10.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.4.1)\.
- \[40\]Y\. Xuan, X\. Chen, Z\. Zhao, B\. Tang, and Y\. Dong\(2023\)Practical and General Backdoor Attacks Against Vertical Federated Learning\.InMachine Learning and Knowledge Discovery in Databases: Research Track,D\. Koutra, C\. Plant, M\. Gomez Rodriguez, E\. Baralis, and F\. Bonchi \(Eds\.\),Vol\.14170,Cham,pp\. 402–417\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-43415-0%5F24),ISBN 978\-3\-031\-43415\-0Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p4.1),[TABLE I](https://arxiv.org/html/2608.12962#S2.T1.5.8.1),[1st item](https://arxiv.org/html/2608.12962#S3.I1.i1.p1.1),[§III\-B2](https://arxiv.org/html/2608.12962#S3.SS2.SSS2.p1.1)\.
- \[41\]Y\. Yan, H\. Wang, Y\. Huang, N\. He, L\. Zhu, Y\. Xu, Y\. Li, and Y\. Zheng\(2024\)Cross\-modal vertical federated learning for MRI reconstruction\.IEEE Journal of Biomedical and Health Informatics28\(11\),pp\. 6384–6394\.External Links:ISSN 2168\-2208,[Document](https://dx.doi.org/10.1109/JBHI.2024.3360720)Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1)\.
- \[42\]Y\. Yang, X\. Chen, Y\. Pan, J\. Shen, Z\. Cao, X\. Dong, X\. Li, J\. Sun, G\. Yang, and R\. Deng\(2024\)OpenVFL: A vertical federated learning framework with stronger privacy\-preserving\.19,pp\. 9670–9681\.External Links:ISSN 1556\-6021,[Document](https://dx.doi.org/10.1109/TIFS.2024.3477924)Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p3.1)\.
- \[43\]M\. Ye, W\. Shen, B\. Du, E\. Snezhko, V\. Kovalev, and P\. C\. Yuen\(2025\)Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey\.ACM Computing Surveys57\(9\),pp\. 1–32\.External Links:ISSN 0360\-0300, 1557\-7341,[Document](https://dx.doi.org/10.1145/3720539)Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p2.1)\.
- \[44\]S\. Yuan, X\. Li, X\. Cao, H\. Zhang, and R\. H\. Deng\(2025\)General Test\-Time Backdoor Detection in Split Neural Network\-Based Vertical Federated Learning\.IEEE Transactions on Dependable and Secure Computing22\(6\),pp\. 7157–7171\.External Links:ISSN 1545\-5971, 1941\-0018, 2160\-9209,[Document](https://dx.doi.org/10.1109/TDSC.2025.3595518)Cited by:[§II\-A](https://arxiv.org/html/2608.12962#S2.SS1.p1.1),[TABLE II](https://arxiv.org/html/2608.12962#S4.T2.5.6.1)\.
- \[45\]F\. Zheng, Erihe, K\. Li, J\. Tian, and X\. Xiang\(2020\)A vertical federated learning method for interpretable scorecard and its application in credit scoring\.External Links:2009\.06218Cited by:[§I](https://arxiv.org/html/2608.12962#S1.p1.1)\.
- \[46\]T\. Zou, Z\. Gu, Y\. He, H\. Takahashi, Y\. Liu, and Y\. Zhang\(2023\)VFLAIR: A Research Library and Benchmark for Vertical Federated Learning\.InThe Twelfth International Conference on Learning Representations,Cited by:[TABLE IV](https://arxiv.org/html/2608.12962#S5.T4.5.5.1)\.

### \-ABenchmark Setup and Evaluation Details

#### \-A1Evaluation Metrics

We report standardized metrics for both attacks and defenses\. Let𝒴=\{1,…,C\}\\mathcal\{Y\}=\\\{1,\\ldots,C\\\}denote the label set, letyty\_\{t\}be the target label of a targeted backdoor attack, and let𝒟c\\mathcal\{D\}\_\{c\}denote the clean test subset whose ground\-truth label iscc\. We writeℱT​\(ℰ𝒙\)\\mathcal\{F\}^\{T\}\(\\mathcal\{E\}\_\{\\bm\{x\}\}\)for the prediction on a clean sample𝒙\\bm\{x\}andℱT​\(ℰ~𝒙\)\\mathcal\{F\}^\{T\}\(\\tilde\{\\mathcal\{E\}\}\_\{\\bm\{x\}\}\)for the prediction when the trigger is injected\. Considering that realistic VFL datasets are often class\-imbalanced, we use macro\-averaged metrics by default so that the majority class does not dominate the calculation\.

Main Task Accuracy \(MTA\)\.We use macro\-F1 on the clean test set as the main\-task utility metric:

MTA=1C​∑c∈𝒴F1c\.\\mathrm\{MTA\}=\\frac\{1\}\{C\}\\sum\_\{c\\in\\mathcal\{Y\}\}\\mathrm\{F1\}\_\{c\}\.\(1\)This definition differs from the conventional metric used in the literature\. Our goal is to provide a more practical evaluation on model utility across all classes rather than mainly on majority classes\.

Attack Success Rate \(ASR\)\.We also define ASR as the macro\-averaged targeted success rate over all source classes:

ASR=1C∑c∈𝒴1\|𝒟c\|∑𝒙∈𝒟c𝟏\[ℱT\(ℰ~𝒙\)=t\]\.\\mathrm\{ASR\}=\\frac\{1\}\{C\}\\sum\_\{c\\in\\mathcal\{Y\}\}\\frac\{1\}\{\|\\mathcal\{D\}\_\{c\}\|\}\\sum\_\{\\bm\{x\}\\in\\mathcal\{D\}\_\{c\}\}\\mathbf\{1\}\[\\mathcal\{F\}^\{T\}\(\\tilde\{\\mathcal\{E\}\}\_\{\\bm\{x\}\}\)=t\]\.\(2\)wherettis the target label,ℰ~𝒙\\tilde\{\\mathcal\{E\}\}\_\{\\bm\{x\}\}is the poisoned embedding of sample𝒙\\bm\{x\}\. We think that this definition prevents biased evaluation on the majority class, as in the conventional definition the attack difficulty on the majority class could significantly skew the overall ASR\. Our definition therefore better reflects whether an attack can consistently redirect predictions from*all*classes to the attacker\-chosen target\.

Label Inference Accuracy\.For attacks that require label inference before poisoning, we explicitly denote the inferred label of sample𝒙\\bm\{x\}byy^𝒙\\hat\{y\}\_\{\\bm\{x\}\}\. Let𝒟inf\\mathcal\{D\}^\{\\mathrm\{inf\}\}be the set of samples on which the attacker uses for further poisoning\. We evaluate label inference quality using macro\-F1, which is aligned with MTA and ASR:

LIA=1C​∑c∈𝒴F1cinf,\\mathrm\{LIA\}=\\frac\{1\}\{C\}\\sum\_\{c\\in\\mathcal\{Y\}\}\\mathrm\{F1\}^\{\\mathrm\{inf\}\}\_\{c\},\(3\)whereF1cinf\\mathrm\{F1\}^\{\\mathrm\{inf\}\}\_\{c\}is the F1 score of classcccomputed from the inferred labels\{y^𝒙∣𝒙∈𝒟inf\}\\\{\\hat\{y\}\_\{\\bm\{x\}\}\\mid\\bm\{x\}\\in\\mathcal\{D\}^\{\\mathrm\{inf\}\}\\\}and the corresponding ground\-truth labels\{y𝒙∣𝒙∈𝒟inf\}\\\{y\_\{\\bm\{x\}\}\\mid\\bm\{x\}\\in\\mathcal\{D\}^\{\\mathrm\{inf\}\}\\\}\.

Stealthiness\.Intuitively, a stealthy attacked sample should still look normal under the clean embedding distribution\. We consider a simple metric: theL∞L\_\{\\infty\}norm of the embedding\. We measure whether the attack pushes a sample into a region with unusually large embedding norm or distance\. To be specific, for a distancerr, we collect its per\-party distribution from training samples, then map each query sample to a scalar valuer⁡\(𝒙\)r\(\\bm\{x\}\)using the CDF of training samples\. We think that samples with close\-to\-one CDF values are more likely to be malicious, while samples whose CDF are small enough are benign\. Then the stealthiness score of the distance measurerris then defined as:

S⁡\(𝒙,C​D​F,r\)=min⁡\(1,2​\(1−C​D​F​\(r⁡\(𝒙\)\)\)\)\.S\(\\bm\{x\};CDF,r\)=\\min\(1,\\,2\(1\-CDF\(r\(\\bm\{x\}\)\)\)\)\.\(4\)This score only penalizes samples whose CDF values are more than 0\.5\. Then we apply this calculation toL​∞L\\inftynorm to form the stealthiness metrics:

r∞​\(𝒙\)=‖𝓔𝒙‖∞r\_\{\\infty\}\(\\bm\{x\}\)=\\\|\\bm\{\\mathcal\{E\}\}\_\{\\bm\{x\}\}\\\|\_\{\\infty\}\(5\)
as theL∞L\_\{\\infty\}norm of the embedding of sample𝒙\\bm\{x\}\.

We provide two ways to collect the reference distribution and evaluate on test samples: oracle mode and label\-free mode\. In oracle mode, we assume that labels of test samples are known, and each party builds class\-conditional CDFsC​D​F∞,cCDF\_\{\\infty,c\}from training samples in classcc\. Therefore, stealthiness scores of test samples can be evaluated from their per\-class reference distribution\. In label\-free mode, we assume that labels of test samples are unknown, and each party builds a single class\-agnostic CDFF∞F\_\{\\infty\}from all training samples\. In this case, stealthiness scores of test samples are evaluated from the global reference distribution instead\.

Overall, oracle mode uses class\-conditional reference distributions and test labels, whereas label\-free mode uses class\-agnostic reference distributions and does not require test labels\. For both modes, we report the per\-sample stealthiness scores of clean and attack test samples in the form of histograms\. We think that the oracle mode provides the most effective evaluation of stealthiness, but in reality test labels are often unavailable, so the label\-free mode is more practical\.

Utility Recovery \(UR\)\.For defenses that aim to recover correct predictions on attacked samples, we report UR as the macro\-F1 of the defended model on the attacked test set:

UR=1C​∑c∈𝒴F1cdef,\\mathrm\{UR\}=\\frac\{1\}\{C\}\\sum\_\{c\\in\\mathcal\{Y\}\}\\mathrm\{F1\}^\{\\mathrm\{def\}\}\_\{c\},\(6\)
whereF1cdef\\mathrm\{F1\}^\{\\mathrm\{def\}\}\_\{c\}is the F1 score of classcccomputed from the defended model’s predictions on the attacked test set and the corresponding ground\-truth labels\. A high UR indicates that the defense can restore the model’s intended functionality on attacked samples\.

Detection Performance\.For defenses that involve test\-time detection, we report the true positive rate \(TPR\) and false positive rate \(FPR\) of their detection results:

TPR=TPTP\+FN,FPR=FPFP\+TN\.\\mathrm\{TPR\}=\\frac\{\\mathrm\{TP\}\}\{\\mathrm\{TP\}\+\\mathrm\{FN\}\},\\qquad\\mathrm\{FPR\}=\\frac\{\\mathrm\{FP\}\}\{\\mathrm\{FP\}\+\\mathrm\{TN\}\}\.\(7\)A high TPR and a low FPR indicate good detection performance, which is an important aspect of defense efficacy\.

#### \-A2Default Configurations

We summarize the training defaults and the dataset\-specific configuration settings used in BVBench in Table[VIII](https://arxiv.org/html/2608.12962#A0.T8)and Table[IX](https://arxiv.org/html/2608.12962#A0.T9), respectively\. Across all experiments, we keep the same training defaults whenever possible, since in practice, the active party cannot tune the hyperparameters for each dataset\. In particular, we use the same training epochs, optimizer with a fixed learning rate and weight decay, and learning\-rate schedule across all settings, and reuse the same default VFL architectures for datasets of the same modality whenever possible\. Dataset\-specific adjustments are restricted to a small set of necessary factors, such as the batch size, loss function, embedding dimension, and the width of the hidden dimensions of the top model, etc\.

TABLE VIII:Shared training defaults in BVBench\.TABLE IX:Dataset\-specific scenario configurations in BVBench\.†\\daggerPTB\-XL uses heterogeneous bottom models in the default setting\.

For attacks and defenses, we mainly follow their default settings in original papers\. Some methods have different sets of hyperparameter choices for different datasets and they do not cover the datasets in BVBench, so we reuse the hyperparameters for CIFAR10 and apply to all datasets\. For hyperparameters not explicitly specified in the paper or source code, we perform a small grid search on CIFAR10 to find the best values, and then fix them across all datasets\.

#### \-A3Extension to Multi\-Target Setting

To extend single\-target attacks to the multi\-target setting, we preserve each method’s original design as much as possible and only replace the single target with all classes in the label space\. Target\-specific prior knowledge is expanded to all target classes without increasing the total number of known labeled samples, and known labels will be proportionally distributed across all classes\. This is to simulate realistic difficulties to obtain those labels\. For attacks with class\-conditioned triggers, this results in one perturbation or trigger module per class\. For attacks with fixed class\-agnostic triggers, we keep the trigger pattern unchanged and instead assign non\-overlapping trigger positions to different classes whenever possible\.