Are Tabular Foundation Models Robust to Realistic Query Distribution Shifts in Microbiome Data?

# Are Tabular Foundation Models Robust to Realistic Query Distribution Shifts in Microbiome Data?
Source: [https://arxiv.org/html/2606.24995](https://arxiv.org/html/2606.24995)
11institutetext:IRD, Sorbonne Université, Unité de Modélisation Mathématique et Informatique des Systèmes Complexes \(UMMISCO\), Paris, France11email:\{giulia\.perciballi, ahmad\.fall, jean\-daniel\.zucker, edi\.prifti\}@ird\.fr22institutetext:Inria, CNRS, I3S, Université Côte d’Azur, Valbonne, France22email:federica\.granese@inria\.fr33institutetext:INSERM, Nutrition et Obésités; Approches Systémiques, NutriOmique, AP\-HP, Hôpital Pitié\-Salpêtrière, Paris, France###### Abstract

Tabular foundation models \(TFMs\) achieve strong performance on microbiome abundance data, yet their robustness under realistic distribution shift remains poorly characterized\. We introduce a benchmark that evaluates the robustness of TFMs to biologically inspired perturbations across six gut microbiome datasets spanning four disease contexts\. In this in\-context learning setting, models receive unperturbed support sets as context and are evaluated on perturbed query samples\. To isolate robustness beyond "shortcut" features, we preserve the most discriminative taxa and apply three controlled perturbation strategies: \(i\) removal of high\-abundance \(uninformative\) taxa, \(ii\) sparsification via increased zero\-inflation, and \(iii\) zero\-imputation via spurious non\-zero injections\. Our results show that protecting discriminative features is insufficient to guarantee stability under support\-query shift: across datasets, all perturbations degrade models’ performances, with zero\-imputation consistently the most harmful, indicating that corrupting global feature structure can break generalization even when key taxa are retained\. Sparsification disproportionately affects TFMs relative to a classical random\-forest baseline, suggesting greater sensitivity to zero\-inflation–type shifts\. We provide the code publicly available at[https://github\.com/UMMISCO/metagenomics\-fm/](https://github.com/UMMISCO/metagenomics-fm/)\.

## 1Introduction

The human microbiome, meaning the collection of microorganisms residing in and on the human body, plays a key role in influencing, directly or indirectly, a broad range of physiological processes in both health and disease\. Consequently, microbiome datasets have become an important testing ground for machine learning methods in biomedical applications\. Microbiome studies characterize microbial communities by profiling the relative abundance of taxa across samples, producing matrices in which rows correspond to samples and columns to taxa \(i\.e\., features\)\. Compared to standard tabular data, microbiome data exhibit two distinctive structural properties:compositionality, since features represent relative abundances that sum to a constant, and pronouncedzero\-inflation, as many taxa are unobserved in a given sample due to biological absence or limited sequencing depth\[[2](https://arxiv.org/html/2606.24995#bib.bib2)\]\. In addition, substantial variability arises across cohorts due to differences in populations, sequencing protocols, and preprocessing pipelines\.

Particularly, in this work, we focus on the gut microbiome, which has been associated with a wide range of systemic diseases, including type 2 diabetes, obesity, inflammatory bowel disease, and liver cirrhosis, and it is known to be particularly diverse compared to other body sites, and its composition varies considerably among healthy individuals\[[21](https://arxiv.org/html/2606.24995#bib.bib21)\]\. These sources of variability make such datasets a challenging setting for machine learning models, particularly when models trained or conditioned on one cohort are applied to samples drawn from different populations or experimental conditions\[[2](https://arxiv.org/html/2606.24995#bib.bib2)\]\.

Tabular foundation models \(TFMs\) have emerged as a powerful alternative to classical machine learning approaches in low\-data regimes, such as those encountered in microbiome data\. These models leverage in\-context learning \(ICL\): at inference time, a labeled support set is provided as context, and predictions are generated in a single forward pass without parameter updates\. Because ICL relies on similarity between support and query examples rather than parameter updates, its effectiveness may be particularly sensitive to distributional mismatch between the contextual examples and the query sample\. Despite their strong performance on standard tabular benchmarks\[[8](https://arxiv.org/html/2606.24995#bib.bib8)\], it is unclear if they are well\-suited to the compositional and sparse structure of microbiome data\. To investigate this issue, we provide the following contributions:

1. 1\.We present an evaluation protocol tailored to TFMs based onquery distribution shift, in which the support set remains unperturbed while perturbations are applied only to query samples\. This design isolates the robustness of in\-context learning under support–query mismatch \(cf\.[Section˜3\.2](https://arxiv.org/html/2606.24995#S3.SS2)\)\.
2. 2\.We develop three biologically inspired perturbation algorithms to modify key structural properties of microbiome abundance data, namely compositionality and zero\-inflation, while preserving informative features\. This design isolates model robustness to distributional shifts without removing the underlying predictive signal \(cf\.[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1)\)\.
3. 3\.Through extensive experiments on six gut microbiome datasets and six state\-of\-the\-art TFMs, we analyze how different forms of distributional perturbations affect tabular foundation models compared to classical baselines, revealing distinct robustness patterns across perturbation types \(cf\.[Section˜5](https://arxiv.org/html/2606.24995#S5)\)\.

### 1\.1Related works

Traditional machine learning models such as XGBoost\[[3](https://arxiv.org/html/2606.24995#bib.bib3)\], Random Forests\[[1](https://arxiv.org/html/2606.24995#bib.bib1)\]and multi\-layer perceptrons have long dominated tabular data classification, but they require task\-specific training and rely heavily on hyperparameters and encoded inductive biases\. To better capture the complex patterns in tabular data, recent approaches have reframed tabular prediction as a table completion problem, where missing entries are predicted from observed ones\. From this perspective, TabPFN\[[10](https://arxiv.org/html/2606.24995#bib.bib10)\]was introduced as the first foundation model for tabular data, leveraging ICL to classify new examples directly from prompts without gradient\-based fine\-tuning\. Building on TabPFN, several models have extended this paradigm to larger datasets and higher\-dimensional feature spaces\. TabICL\[[20](https://arxiv.org/html/2606.24995#bib.bib20)\]uses a factorized transformer with column\-wise and row\-wise attention to scale ICL efficiently\. TabDPT\[[14](https://arxiv.org/html/2606.24995#bib.bib14)\]combines retrieval\-augmented transformers with Flash Attention to handle real\-world tabular datasets without dataset\-specific tuning\. ContextTab\[[22](https://arxiv.org/html/2606.24995#bib.bib22)\]incorporates semantic embeddings of column names to leverage feature meanings in large\-scale tables\. These models demonstrate how the table\-completion framing enables transformer\-based foundation models to generalize across diverse tabular tasks while avoiding per\-task retraining\. Several extensions have been built on this idea\. TabPFN\-Wide\[[12](https://arxiv.org/html/2606.24995#bib.bib12)\]and TabFLEX\[[24](https://arxiv.org/html/2606.24995#bib.bib24)\]apply ICL to wider and more complex tables, while newer versions of TabICL and TabPFN \([Section˜4\.1\.4](https://arxiv.org/html/2606.24995#S4.SS1.SSS4)\) further improve efficiency and feature coverage through factorized transformer architectures and optimized prompts\. Among domain\-specific applications, the authors in\[[18](https://arxiv.org/html/2606.24995#bib.bib18)\]have, instead of modifying the architecture of TabPFN, adapted the prior of PFN models to generate metagenomic\-like synthetic data\.

## 2Preliminaries

### 2\.1Microbiome abundance data

Microbiome abundance data is represented as a matrix \(a taxonomic abundance table\)𝐗∈ℝn×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{n\\times d\}, where rows represent samples, columns represent taxa \(i\.e\., the features from a machine learning perspective\), and each entry𝐗i​j\\mathbf\{X\}\_\{ij\}denotes the abundance of taxonjjin sampleii\. This data type exhibits two structural properties that distinguish it from generic tabular data\[[2](https://arxiv.org/html/2606.24995#bib.bib2)\]:

1. 1\.Compositionality\. In taxonomic abundance tables, the individual components of a sample sum to a constant, and consequently, the features lie on a simplex, which means increasing the abundance of one taxon necessarily decreases the relative contribution of others\. Formally,𝐗\\mathbf\{X\}is a compositional set if and only if∀i∈\{1,…,n\}\\forall i\\in\\\{1,\\dots,n\\\}the vector row𝐱i\\mathbf\{x\}\_\{i\}of𝐗\\mathbf\{X\}is in the simplex 𝒞d=\{𝐱i∈ℝd:∀j,xi​j\>0;∑j=1dxi​j=κ\},\\mathcal\{C\}^\{d\}=\\left\\\{\\mathbf\{x\}\_\{i\}\\in\\mathbb\{R\}^\{d\}\\,:\\,\\forall j,x\_\{ij\}\>0;\\sum\_\{j=1\}^\{d\}x\_\{ij\}=\\kappa\\right\\\},whereκ\>0\\kappa\>0is a constant, generally 1\.
2. 2\.Sparsity\. Taxonomic abundance tables have a high percentage of zero values, implying right\-skewed distributions with considerable point mass at zero \(zero\-inflation\)\. Notably, zeros should be treated with caution, as they may result either from the biological heterogeneity of microbial communities, in which many organisms are detected in only a few samples, or from technical limitations that prevent the detection of some low\-abundance taxa\.

The two above properties directly motivate the algorithms in[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1)\. Compositionality implies that locally perturbing one component, i\.e\., a taxon, directly impacts all other components of the sample, motivating our Algorithm 1\) and Algorithm 3\)\. The zero\-inflation structure makes models sensitive to perturbations that alter the zero/non\-zero pattern of the samples, motivating our Algorithm 2\)\.

### 2\.2Tabular foundation models

TFMs are a new class of neural networks tailored for tabular data\. Their outstanding performance stems from performing predictions viaICL: at inference time, a labeledsupport set𝒮\\mathcal\{S\}is provided as context, and the model produces predictions for unseenquery samples𝒬\\mathcal\{Q\}in a single forward pass, without any parameter updates\. TFMs are based on the Prior\-Data Fitted Networks \(PFNs\) framework\[[16](https://arxiv.org/html/2606.24995#bib.bib16)\], in which a transformer is pre\-trained on synthetic datasets sampled from a prior over tasks\. This process meta\-learns an approximation to Bayesian posterior predictive inference, allowing the model to generalize to new tasks at test time, provided the target distribution is consistent with the pretraining prior\.

TFMs have shown strong performance across a wide range of use cases\. However, their robustness on microbiome abundance data remains poorly understood\. While a previous work\[[18](https://arxiv.org/html/2606.24995#bib.bib18)\]explored adapting the prior distribution of TabPFN\[[10](https://arxiv.org/html/2606.24995#bib.bib10)\]to this data type, it did not systematically analyze the failure modes of TFMs in this domain\. Furthermore, that work preceded the recent proliferation of TFMs, and more capable models are now available\. This motivates our systematic robustness evaluation of TFMs on microbiome data\.

### 2\.3TFM pretraining

The pretraining regime fundamentally shapes TFMs’ behavior at inference time\. TabPFN and TabICL are both pretrained on synthetic datasets sampled from a structured prior over tasks, and consequently carry no implicit prior over real feature distributions\. However, TabICL introduces a column\-wise Set Transformer that is explicitly distribution\-aware with respect to the support set\. ConTextTab and TabDPT, on the contrary, utilize large\-scale pretraining on real\-world data, T4\[[6](https://arxiv.org/html/2606.24995#bib.bib6)\]and OpenML\[[23](https://arxiv.org/html/2606.24995#bib.bib23)\]datasets, respectively\. As a result, they develop priors over realistic data distributions\.

![Refer to caption](https://arxiv.org/html/2606.24995v1/images/pipeline_perturbation.png)Figure 1:Perturbation pipeline– Starting from the raw taxonomic abundance matrix𝐗\\mathbf\{X\}, informative featuresℱI\\mathcal\{F\}\_\{I\}are identified via ANOVA F\-test and Random Forest and protected from perturbation\. One of three perturbation algorithms is then applied exclusively to the uninformative featuresℱU\\mathcal\{F\}\_\{U\}\. The perturbed matrix𝐗′\\mathbf\{X\}^\{\\prime\}is finally reconstructed by concatenatingℱU′\\mathcal\{F\}\_\{U\}^\{\\prime\}with the originalℱI\\mathcal\{F\}\_\{I\}, followed by row\-wise renormalization to preserve the compositional constraint\.

## 3Method

This section describes the proposed framework to assess the robustness of TFMs on microbiome abundance data\. We first introduce our biologically inspired perturbation strategies, and then we describe the evaluation protocol\. Additional details on the experimental parameters are provided in[Section˜4\.1](https://arxiv.org/html/2606.24995#S4.SS1)\.

### 3\.1Perturbation pipeline

The goal of the perturbation pipeline \(cf\.[Figure˜1](https://arxiv.org/html/2606.24995#S2.F1)\) is to design a framework introducing controlled distribution shifts targeting the two structural properties of microbiome data: compositionality and sparsity \(i\.e\., zero\-inflation\)\. In what follows, we will call𝐗′\\mathbf\{X\}^\{\\prime\}the taxonomic abundance matrix after perturbations\. Notably, in our algorithms, we require that𝐗′\\mathbf\{X\}^\{\\prime\}remains compositional\. We relegate in Section A\.2 \(of supplementary materials\) the pseudocode of the proposed perturbation algorithms\. We provide the mean abundance distributions of individual features across increasing perturbation levels, stratified by class in Figure 5 \(of supplementary materials\)\.

#### 3\.1\.1Informativevs\.uninformativefeature selection\.

To avoid trivial failure modes, we first distinguishinformativefromuninformativefeatures\. Specifically, we define a feature asinformativeif it contributes significantly to the predictive performance of a model on the task of interest\. We note that features deemed important by a machine learning model may not coincide with those considered relevant by a domain expert\. However, since our goal is to assess model robustness to query distribution shift, we adopt a data\-driven notion of feature importance\. Perturbing highly discriminative taxa would trivially degrade performance and therefore provide limited insight into robustness under distribution shift\. Our perturbations are therefore applied exclusively to uninformative features, isolating robustness beyond reliance on shortcuts\.

Let𝐲∈𝒴n\\mathbf\{y\}\\in\\mathcal\{Y\}^\{n\}denote the vector of target labels for the task of interest\. We first rank features by importance via one\-way ANOVA F\-tests, which provides a model\-agnostic ordering without requiring a secondary learner\. To determine the number of truly informative features in a dataset, we apply the following procedure\. A Random Forest classifier is trained on the full dataset under cross\-validation, yielding a baseline AUROCa0a\_\{0\}\. We then iteratively remove the top\-ranked feature and recompute the AUROCaja\_\{j\}after each removal stepjj\. As long asaj−1−aj<δa\_\{j\-1\}\-a\_\{j\}<\\delta\(i\.e\., the drop is below thresholdδ=0\.03\\delta=0\.03\), the removed feature is considered informative\. The process stops at the first stepj⋆j^\{\\star\}where the drop exceedsδ\\delta, and we defineℱI=\{1,…,j⋆\}\\mathcal\{F\}\_\{I\}=\\\{1,\\dots,j^\{\\star\}\\\}as the set of informative features andℱU=\{j⋆\+1,…,d\}\\mathcal\{F\}\_\{U\}=\\\{j^\{\\star\}\+1,\\dots,d\\\}as the uninformative ones\. Indeed, microbiome abundance data are known for their high heterogeneity across datasets\[[2](https://arxiv.org/html/2606.24995#bib.bib2)\]\. Fixing the number of informative features a priori \(e\.g\., always retaining the top\-5 ranked features\) would introduce evaluation bias: performance differences across datasets could then reflect the arbitrary choice of cutoff rather than true differences in model robustness \(cf\. Figure 6 of supplementary materials\)\. By keepingℱI\\mathcal\{F\}\_\{I\}fixed, perturbations target only uninformative features, ensuring that performance differences reflect the model’s sensitivity to distributional shift rather than loss of essential signal\.

#### 3\.1\.2Perturbations algorithms

Given the setℱU\\mathcal\{F\}\_\{U\}and the taxonomic abundance matrix𝐗\\mathbf\{X\}, we consider the following perturbation strategies\.

##### Feature Removal\(Algorithm 1\.\)

It evaluates model robustness to changes in feature availability, a common reason of variability in microbiome data, where differences in sequencing protocols can result in certain taxa being observed in some samples but absent in others\[[5](https://arxiv.org/html/2606.24995#bib.bib5)\]\. We remove thekkuninformative features with the highest mean abundance across samples\. This creates a challenging scenario because, although uninformative for classification, high\-abundance taxa contribute substantially to the data’s compositional structure\. The renormalization of the rows after removal of such taxa systematically inflates the relative abundances of the remaining ones, shifting their marginal distributions and altering the global compositional structure\. This perturbation strategy most directly stresses the compositional structure of microbiome data\.

##### Zero Inflation\(Algorithm 2\.\)

It directly targets the zero\-inflation structure of microbiome data, simulating the variability in sparsity patterns that arises from technical differences in how microbiome samples are processed and sequenced\. Given a target sparsity levelρ⋆\\rho^\{\\star\}, we compute the number of zeroskkto introduce intoℱU\\mathcal\{F\}\_\{U\}ask=min⁡\{ρ⋆−ρ,ϕU\}k=\\min\\\{\\rho^\{\\star\}\-\\rho,\\,\\phi\_\{U\}\\\}, whereρ\\rhois the current number of zeros,ϕU\\phi\_\{U\}is the number of non\-zero entries available inℱU\\mathcal\{F\}\_\{U\}\. We then apply power transformation

x~i​j=\(xi​j\)1\+γ⋅𝟙​\[\(xi​j\)1\+γ≥τ\],j∈ℱU,\\displaystyle\\tilde\{x\}\_\{ij\}=\(x\_\{ij\}\)^\{1\+\\gamma\}\\cdot\\mathds\{1\}\\left\[\\,\(x\_\{ij\}\)^\{1\+\\gamma\}\\geq\\tau\\right\],\\qquad j\\in\\mathcal\{F\}\_\{U\},\(1\)whereγ\>0\\gamma\>0controls the degree of shrinkage andτ=10−6\\tau=10^\{\-6\}is a fixed detection threshold\.[Equation˜1](https://arxiv.org/html/2606.24995#S3.E1)preferentially suppresses low\-abundance taxa while preserving the relative ordering among dominant features\. To reach the target zero countkk, we exploit the fact that larger values ofγ\\gammaproduce more aggressive shrinkage and therefore more zeros\. We search for theγ\\gammathat minimizes\|ρU′−k\|\|\\rho\_\{U\}^\{\\prime\}\-k\|via binary search, whereρU′\\rho\_\{U\}^\{\\prime\}counts the newly introduced zeros at each iteration\. The best iterate is retained if exact convergence is not achieved withinTTiterations\. After transformation, rows are renormalized to restore compositionality, inflating the relative abundances of the remaining taxa and altering the global compositional structure\. This perturbation, therefore, jointly tests robustness to increased zero\-inflation and to compositional redistribution\.

##### Zero Imputation\(Algorithm 3\.\)

It directly targets the zero\-inflation structure of microbiome data, but in the opposite direction toZero Inflation: rather than introducing new zeros, it fills existing ones\. Given a target sparsity levelρ⋆\\rho^\{\\star\}, we computek=ρ−ρ⋆k=\\rho\-\\rho^\{\\star\}, whereρ\\rhois the current number of zeros, as the number of zeros to fill, capped at\|𝒵\|\|\\mathcal\{Z\}\|, the number of zero entries available inℱU\\mathcal\{F\}\_\{U\}\. We randomly selectkkpositions from𝒵\\mathcal\{Z\}and replace each zero entry\(i,j\)\(i,j\)with a value drawn uniformly from𝒱j\\mathcal\{V\}\_\{j\}, the set of observed non\-zero values of featurejjacross all samples\. This ensures that filled values are not synthetic noise but plausible abundance levels already observed in the dataset for that taxon\. If featurejjhas no observed non\-zero values, a fallback feature is selected uniformly at random fromℱU\\mathcal\{F\}\_\{U\}\. After filling, rows are renormalized to restore compositionality\. As with the previous algorithms, this renormalization alters the global compositional structure beyond the mere injection of non\-zero values, jointly analyzing robustness to reduced zero\-inflation and to compositional redistribution\.

### 3\.2Evaluation pipeline

#### 3\.2\.1Query distribution shift

We evaluate TFMs under a setting we namequery distribution shift: the support set𝒮\\mathcal\{S\}is always drawn from the original unperturbed data while the query set𝒬\\mathcal\{Q\}is replaced by its perturbed counterpart𝒬′\\mathcal\{Q\}^\{\\prime\}obtained via the algorithms we define in[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1)\. The distribution shift is therefore defined with respect to the support, and its magnitude is controlled by the parameter of the perturbations \(e\.g\.,kk,ρ⋆\\rho^\{\\star\},τ\\tau\)\.

This setting differs from two related notions: adversarial attacks\[[7](https://arxiv.org/html/2606.24995#bib.bib7)\]and out\-of\-distribution \(OOD\) generalization\[[9](https://arxiv.org/html/2606.24995#bib.bib9)\]\. In adversarial settings, perturbations are typically optimized to maximize model error under norm constraints and need not correspond to biologically meaningful transformations of the data\. In contrast, OOD generalization studies scenarios where the test distribution differs from the distribution used to train the model\. In our ICL setting, however, there is no task\-specific training phase: the model receives𝒮\\mathcal\{S\}as context at inference time and must generalize to𝒬′\\mathcal\{Q\}^\{\\prime\}in a single forward pass\. As a result, the distribution shift arises between the support and query sets rather than between training and test distributions\.

#### 3\.2\.2Why perturbing only the query

Each perturbation family is designed to approximate a distinct source of support\-query mismatch that may arise in clinical microbiome datasets\.Feature Removalcaptures scenarios in which features that are observed in the reference cohort are absent in the new samples, for instance, due to differences in sequencing depth, measurement procedures, or preprocessing pipelines\.Zero Inflationmodels an increase in zero\-inflation at query time, reflecting settings where query samples contain fewer observable taxa or where the measurement process produces sparser observations\.Zero Imputationrepresents the opposite situation, where features absent in the support set become observable in the query, which may occur when models are applied across cohorts with higher measurement sensitivity or processed with different protocols\. In all three cases, the informative featuresℱI\\mathcal\{F\}\_\{I\}are preserved, ensuring that the discriminative signal underlying the task remains available, and that performance differences can be attributed to the distributional mismatch rather than to the removal of task\-relevant information\.

## 4Experiments

The proposed framework comprises6​datasets×6​models×3​perturbation families×15​perturbation levels×5​CV folds=8,1006\\ \\text\{datasets\}\\times 6\\ \\text\{models\}\\times 3\\ \\text\{perturbation families\}\\times 15\\ \\text\{perturbation levels\}\\times 5\\ \\text\{CV folds\}=8\{,\}100evaluations in total\.

### 4\.1Experimental setting

#### 4\.1\.1Datasets

We consider publicly curated metagenomic datasets from ExperimentHub\[[17](https://arxiv.org/html/2606.24995#bib.bib17)\], which includes six cohorts with data at six taxonomic levels\. We focused on the species\-level taxa \(microbial species relative abundance\) across five cohorts: Cirrhosis \(discovery: 178 samples, 83/95 control/case, 520 features; validation: 54 samples, 31/23, 394 features\), Inflammatory Bowel Disease \(IBD; 110 samples, 85/25, 443 features\), Obesity \(253 samples, 89/164, 465 features\), Type 2 Diabetes in Chinese patients \(T2D; 344 samples, 174/170, 572 features\), and Type 2 Diabetes in European women \(WT2D; 96 samples, 43/53, 381 features\)\.

For each cohort, the task is binary classification\. No preprocessing was applied to any dataset\.

#### 4\.1\.2Perturbation pipeline parameter setting

We described the selection process for informative and uninformative features in[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1)\. Specifically, we perform 5\-fold stratified cross\-validation and set theδ\\deltathreshold toa0−aj<δa\_\{0\}\-a\_\{j\}<\\delta= 0\.03, meaning the AUROC degrades by at most 3 percentage points\. This threshold represents a reasoned balance: a5%5\\%degradation would suggest that a substantial biological signal has been removed, while a more stringent threshold would protect very few features, providing insufficient sensitivity to perturbations\. Additional discussion is provided in Section C\.1 \(of supplementary materials\)\.

#### 4\.1\.3Perturbation algorithms parameter setting

We described the perturbation algorithms in[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1)\. In particular because of the heterogeneity of microbiome datasets, we set the perturbation parameters in a data\-dependent manner, anchored to the original sparsityρ\\rhoand the number of taxaddof each dataset\.

Algorithm 1:the number of removed featureskkranges over a grid of 10 evenly spaced values between 1 tod/2d/2\.

Algorithm 2:the target sparsity is varied over a grid of 5 evenly spaced values between the original sparsityρ\\rhoand0\.990\.99\(we increase the number of zeros\)\. The binary search runs for at mostT=50T=50iterations withγmin=0\\gamma\_\{\\min\}=0,γmax=10\\gamma\_\{\\max\}=10, and detection thresholdτ=10−6\\tau=10^\{\-6\}\. The upper boundγmax=10\\gamma\_\{\\max\}=10ensures sufficient shrinkage coverage: for any relative abundancex<10−6/11≈0\.285x<10^\{\-6/11\}\\approx 0\.285, we havex11<τ=10−6x^\{11\}<\\tau=10^\{\-6\}, so the entry is set to zero\. Since microbiome abundance data are compositional and our datasets contain hundreds of taxa, the vast majority of non\-zero values fall below this threshold in practice\. Moreover, when the number of available non\-zero entries inℱU\\mathcal\{F\}\_\{U\}is insufficient to reach the target, i\.e\.ϕU<ρ⋆−ρ\\phi\_\{U\}<\\rho^\{\\star\}\-\\rho, the algorithm clipskktoϕU\\phi\_\{U\}, and the resulting sparsity will be lower thanρ⋆\\rho^\{\\star\}\. In our experiments, this situation does not arise, as all datasets contain sufficiently many non\-zero entries inℱU\\mathcal\{F\}\_\{U\}relative to the perturbation targets\. Finally, the algorithm terminates early once\|ρU′−k\|≤1\|\\rho\_\{U\}^\{\\prime\}\-k\|\\leq 1, i\.e\., when the number of newly introduced zeros differs from the target by at most one entry\. This condition is met well beforeTTiterations in all our experiments, confirming thatT=50T=50acts as a reasonable upper bound on the number of iterations\.

Algorithm 3:the target sparsity is varied over a grid of 5 evenly spaced values between0\.010\.01andρ\\rho\(we reduce the number of zeros\)\.

#### 4\.1\.4TFMs

We evaluate four TFMs together with two classical machine\-learning baselines: Random Forest \(RF\)\[[1](https://arxiv.org/html/2606.24995#bib.bib1)\]and XGBoost\[[3](https://arxiv.org/html/2606.24995#bib.bib3)\], both configured with 100 estimators, as detailed in[Section˜3\.2](https://arxiv.org/html/2606.24995#S3.SS2)\. All methods are assessed using 5\-fold stratified cross\-validation to ensure that class proportions are preserved across folds\. As for the TFMs, we consider the following models introduced in[Section˜1\.1](https://arxiv.org/html/2606.24995#S1.SS1)\. TabPFN\[[8](https://arxiv.org/html/2606.24995#bib.bib8),[10](https://arxiv.org/html/2606.24995#bib.bib10),[11](https://arxiv.org/html/2606.24995#bib.bib11)\]\(v6\.3\.2\) is run with its default settings, including the 3\-model ensemble and no gradient computation\. TabICL\[[19](https://arxiv.org/html/2606.24995#bib.bib19),[20](https://arxiv.org/html/2606.24995#bib.bib20)\]\(v2\.0\.1\) is also used with default parameters, with 32 estimators for ensemble predictions\. TabDPT\[[14](https://arxiv.org/html/2606.24995#bib.bib14)\]\(v1\.1\.12\) is included as well with 8 estimators as default; this model supports up to 100 features, applying PCA\-based reduction when needed\. ContextTab\[[22](https://arxiv.org/html/2606.24995#bib.bib22)\]\(v1\.0\.1\) is used with acontext\_sizeof 2048 and abagging\_factorof 1\.

## 5Results

We assess model robustness using the algorithms in[Section˜3\.1](https://arxiv.org/html/2606.24995#S3.SS1), which simulate distribution shifts commonly faced when deploying microbiome classifiers across cohorts, sequencing platforms, or clinical sites\.[Figure˜2\(a\)](https://arxiv.org/html/2606.24995#S5.F2.sf1)summarizes the mean AUROC degradation across all datasets and perturbations, and[Figure˜2\(b\)](https://arxiv.org/html/2606.24995#S5.F2.sf2)characterizes the tradeoff between baseline discriminative performance and robustness\. Baseline AUROC values are reported in Table 2 \(of supplementary materials\)\. Throughout this section, all significance tests use the DeLong test\[[4](https://arxiv.org/html/2606.24995#bib.bib4)\]atα=0\.05\\alpha=0\.05, and we denote bynsign\_\{\\text\{sig\}\}the number of model–dataset–perturbation\-level combinations reaching significance\. Additional results in Section C\.3 \(of supplementary materials\)\.

![Refer to caption](https://arxiv.org/html/2606.24995v1/x1.png)\(a\)AUROC degradation under perturbation– Mean change in AUROC \(percentage points\) relative to baseline for each model across datasets, grouped by perturbation\. Error bars show ±1 standard error across cross\-validation folds and perturbation levels\. Asterisks denote model–dataset–perturbation combinations where at least one perturbation level had a statistically significant AUROC change\(DeLong test, p < 0\.05\)\.
![Refer to caption](https://arxiv.org/html/2606.24995v1/x2.png)\(b\)Robustness–accuracy tradeoff– Relationship between baseline AUROC and mean AUROC change under perturbation for each model–dataset combination\. Each point represents one model on one dataset\. Error bars show ±1 standard error on both axes\. Points near the dashed line\(y = 0\)are robust, points in the upper\-right quadrant combine high baseline accuracy with minimal degradation\.

Figure 2:Model robustness under compositional perturbations\.\(a\) AUROC degradation by model and perturbation type\. \(b\) Baseline accuracy versus robustness tradeoff per perturbation type\.#### 5\.0\.1Feature Removal

Removing the most abundant \(uninformative\) features has a generally mild effect on AUROC \([Figure˜2\(a\)](https://arxiv.org/html/2606.24995#S5.F2.sf1), green bars\), which is expected given the high taxonomic redundancy of microbiome profiles\. The most significant results fall under IBD, where all models degrade monotonically with the number of removed featureskk\. This suggests that the discriminative signal in IBD is spread across many taxa rather than carried by a few dominant species\.TabDPTshows the steepest drop on IBD \(−13\-13pp\), followed byRFandTabPFN\. OnlyContextTabandTabPFNon Cirrhosis 2 show sensitivity to feature removal\.

Interestingly, removal can also*help*\. Both tree\-based models improve on WT2D, with mean gains of\+7\.4\+7\.4pp \(RF\) and\+5\.9\+5\.9pp \(XGBoost\) across perturbation levels, reaching up to\+11\.0\+11\.0pp and\+11\.6\+11\.6pp, respectively, at individual levels \(p<0\.05p<0\.05\)\. This is consistent with overfitting to noisy high\-abundance features that the perturbation eliminates\. ForXGBoost, this is the only significant result under feature removal across all datasets, suggesting that split\-based feature selection already provides implicit regularisation\.TabDPTshows a similar trend on datasets with more features than its fixed input capacity \(dfeatd\_\{\\text\{feat\}\}\), where removing features reduces the need to compress the input and may preserve more signal, though the gains do not reach significance\. These patterns are also visible at the individual prediction level\. The flip rate \([Figure˜3](https://arxiv.org/html/2606.24995#S5.F3), solid lines\) rises steeply for IBD, reaching∼55%\{\\sim\}55\\%of predictions changed at maximum perturbation, whereas Cirrhosis 1 stays below 15% for all models, confirming the stability of its discriminative signal\. This contrast may also be partly amplified by class balance: IBD’s pronounced imbalance \(85/25\) implies as few as∼5\{\\sim\}5minority\-class samples per test fold under 5\-fold stratified CV, whereas Cirrhosis 1 is nearly balanced \(83/95\), yielding comparatively more stable per\-fold estimates \(Table 2 of supplementary materials\)\. Overall, with respect toFeature Removal, the results suggest that robustness is primarily determined by the structure of the discriminative signal in each dataset rather than by model architecture\.

![Refer to caption](https://arxiv.org/html/2606.24995v1/x3.png)Figure 3:Prediction flip rate under perturbation– Fraction of test samples whose predicted class changes relative to baseline as a function of normalised perturbation intensity\(0 = unperturbed, 1 = maximum perturbation\)\. A higher flip rate indicates greater decision instability — the model would assign a different classification to the same patient depending on data quality\.
#### 5\.0\.2Zero Imputation

While feature removal affects models moderately, zero imputation is far more damaging as it fills structural zeros with realistic non\-zero values\. No model improves under any condition; all significant results are degradations\.[Figure˜2\(b\)](https://arxiv.org/html/2606.24995#S5.F2.sf2)\(center panel\) shows the widest vertical spread of the three perturbation types, reflecting large differences in how models respond\.

TabDPTstands out with 23 significant degradations, losing up to−31\.6\-31\.6pp on Cirrhosis 1 and−32\.1\-32\.1pp on Cirrhosis 2\. In the scatter plot, its points lie well below those of all other models, regardless of baseline AUROC\.TabDPToperates as an in\-context learner: at inference time, it conditions its predictions on a set of training examples \(the context\) passed alongside the test sample\. The transformer’s attention mechanism implicitly retrieves the most relevant context examples to inform its output\. BecauseTabDPTwas pre\-trained on data with many structural zeros, this retrieval works best when test inputs are similarly sparse\. Zero imputation fills those zeros with realistic non\-zero values, making the test input look fundamentally different from the context examples the model relies on\. The attention mechanism can no longer match the test sample to relevant training patterns, resulting in large performance drops\.

TabICLis the next most affected \(nsig=16n\_\{\\text\{sig\}\}=16\)\. Its column\-wise Set Transformer estimates the distribution of each feature across samples and uses these per\-feature summaries to build row\-level representations\. As more zeros are replaced with imputed values, the observed feature distributions shift further from the true ones, andTabICL’s representations become increasingly unreliable, which explains why its degradation grows monotonically with the fraction of imputed values\.ContextTab\(nsig=12n\_\{\\text\{sig\}\}=12\) degrades mainly on IBD, Obesity, and T2D, whileRF,TabPFN, andXGBoostare affected more selectively\.

From the dataset side, T2D is the most vulnerable cohort \(nsig=25n\_\{\\text\{sig\}\}=25, all six models degrade significantly\), followed by IBD \(nsig=17n\_\{\\text\{sig\}\}=17\)\. Cirrhosis 1, by contrast, yields only 6 significant degradations, all fromTabDPT, and sits in the upper\-right quadrant of the scatter plot\. Its high density of informative features likely buffers against the introduction of spurious non\-zero values\.

[Figure˜3](https://arxiv.org/html/2606.24995#S5.F3)\(dotted lines\) confirms how severe this perturbation is at the sample level:TabDPTexceeds 50% on Cirrhosis 2, and evenXGBoost, the most stable model overall, reaches 15–25% on IBD and T2D\. This shows that zero imputation affects not only aggregate metrics but also individual patient\-level decisions\.

This perturbation targets a central ambiguity of metagenomic data: zeros may denote true biological absence or technical dropout driven by sequencing depth or protocol differences\[[13](https://arxiv.org/html/2606.24995#bib.bib13)\]\. Our results, however, indicate that the observed degradation can be primarily attributable to model\-side weaknesses rather than to this biological ambiguity: RF and XGBoost remain substantially more robust under zero imputation \([Figure˜2\(a\)](https://arxiv.org/html/2606.24995#S5.F2.sf1)\), while TabDPT, TabICL, and ContextTab lose up to 30\+ AUROC points\. This asymmetry points to an architecture\-specific reliance on the distributional structure ofℱU\\mathcal\{F\}\_\{U\}\- for instance, attention\-based retrieval or per\-feature distribution estimation, as the primary driver of the observed fragility \- though ruling out alternative explanations entirely would require additional analyses motivated by the limitations of Zero Imputation discussed in \([Section˜7](https://arxiv.org/html/2606.24995#S7)\)\.

#### 5\.0\.3Zero Inflation

This perturbation increases sparsity by replacing non\-zero abundance values with zeros\.TabDPTis the only model to improve significantly \(nsig=11n\_\{\\text\{sig\}\}=11\): up to\+19\.7\+19\.7pp on WT2D,\+13\.6\+13\.6pp on Cirrhosis 2, and\+8\.8\+8\.8pp on T2D, while all other models degrade moderately\. In the scatter plot,TabDPTpoints are the only ones consistently above the zero line, confirming that increasing sparsity moves the input towards the sparse patterns of its training input, improving retrieval quality\.

Conversely, the models that were relatively resilient to zero imputation now degrade\.ContextTabloses up to−12\.1\-12\.1pp on Obesity \(nsig=4n\_\{\\text\{sig\}\}=4\) and−11\.0\-11\.0pp on IBD, whileTabICLdrops up to−9\.9\-9\.9pp on Obesity \(nsig=3n\_\{\\text\{sig\}\}=3\),−11\.2\-11\.2pp on Cirrhosis 2, and−10\.0\-10\.0pp on WT2D\. Despite differing architectures, both models share a key property: they have learned realistic abundance distributions during pre\-training\.TabICLdoes so through its column\-wise Set Transformer, which builds feature\-distribution\-aware embeddings,ContextTabrelies on column\-name embeddings that couple feature identity with abundance patterns\. Zero inflation destroys precisely these learned distributions by replacing informative values with zeros\.TabPFN, despite sharing a dual\-attention architecture withContextTab, is comparatively resilient – trained exclusively on synthetic data, it carries no prior on real feature distributions\. Tree\-based models degrade modestly and only on specific datasets\. Cirrhosis 1, as under feature removal, shows no significant degradation for any model\.

Flip rates \([Figure˜3](https://arxiv.org/html/2606.24995#S5.F3), dashed lines\) vary strongly across datasets\. IBD reaches∼85%\{\\sim\}85\\%forContextTabat maximum perturbation, while T2D and Obesity remain below 30% for most models, likely reflecting differences in the fraction of biologically meaningful non\-zero entries\. This perturbation, therefore, shows a genuine fragility: models are sensitive to changes in abundance values that should carry negligible discriminative weight\.

#### 5\.0\.4Cross\-model shift concordance

![Refer to caption](https://arxiv.org/html/2606.24995v1/x4.png)Figure 4:Pairwise prediction shift concordance– Distribution of Spearman correlations between per\-sample prediction shifts across all model pairs, faceted by perturbation type\. Each point represents a dataset at a given perturbation level\. High positive correlations indicate that both models are affected in the same direction on the same samples\(shared vulnerabilities\), low or negative correlations indicate complementary failure modes, favorable for ensembling\.The results above do not reveal whether models fail on the same samples\. To answer this question, we compute pairwise Spearman correlations of per\-sample prediction shifts across all perturbation levels and datasets \([Figure˜4](https://arxiv.org/html/2606.24995#S5.F4)\)\. The results indicate a clear stratification\. ICL\-based foundation model pairs \(ContextTab–TabICL,ContextTab–TabPFN,TabICL–TabPFN\) and theContextTab–RFpair show the highest concordance \(medianρ≈0\.6\\rho\\approx 0\.6–0\.850\.85\) across all three perturbation types: when one model shifts its prediction on a sample, the other shifts in the same direction\. These models share failure modes at the instance level\. All pairs involvingTabDPTcluster at the bottom, with median correlations near zero and sometimes negative under zero inflation\. This meansTabDPTfails on different samples and often in the opposite direction compared to other models\. Despite its fragility under zero imputation, this complementarity makes it a valuable candidate for ensemble strategies\. Importantly, this concordance structure is stable across perturbations: pairs that are concordant under feature removal remain concordant under zero imputation and zero inflation\. This suggests that the similarity of failure patterns reflects shared architectural priors, in particular the ICL paradigm, rather than properties of the perturbation\. In practice, the ensembling potential of a model pair can therefore be assessed from a single perturbation type\.

## 6Conclusions

We proposed a framework to evaluate TFMs on microbiome data under realistic query distribution shifts\. Using three biologically motivated perturbation strategies:Feature Removal,Zero InflationandZero Imputation, applied across six gut microbiome datasets and six models, we characterized how structural properties unique to microbiome data challenge in\-context learners in ways that standard benchmarks do not reveal\.

We report three main findings\. First,no model is robust across all perturbation types: every model degrades significantly under at least one condition, and the pattern of vulnerabilities differs across architectures\.Zero Imputationis consistently the most harmful perturbation, causing widespread degradation across all models and datasets, whereasFeature Removalhas comparatively mild, dataset\-dependent effects\. Second,protecting discriminative features is insufficient to guarantee stability: even when informative taxa are fully preserved, corrupting the global compositional and sparsity structure of query samples degrades in\-context learning, indicating that TFMs rely on global feature structure beyond the discriminative signal itself\. Third,there is a tradeoff between semantic richness and distributional robustness\. Models that have learned realistic abundance distributions during pre\-training – such asTabICLandContextTab– achieve stronger baseline performance but are disproportionately sensitive to zero inflation type shifts\. Conversely,TabPFNexhibits greater robustness to sparsity perturbations at the cost of richer representational capacity\.TabDPToccupies a complementary niche: severely vulnerable to zero imputation yet uniquely benefiting from increased sparsity, with failure modes largely uncorrelated with other models\.

From a practical standpoint, dataset properties matter as much as model choice\. Models on the Cirrhosis 1 dataset remain robust under all perturbations, unlike in T2D and IBD, suggesting that the density and redundancy of informative features contribute to robustness\. When sparsity patterns or cohort composition may differ between training and deployment, as is common in multi\-site microbiome studies, simpler models or architecturally diverse ensembles may offer better reliability than any single TFM\. Our findings also raise questions about interpretability and clinical relevance, as model\-based feature importance may not reflect biological function\. Taxa considered uninformative by our criterion may still be clinically meaningful, while model\-relied features may capture cohort artefacts rather than true biological signal\.

## 7Future Work

Improving robustness\.Our findings motivate several directions for future work\. The most immediate is to move from characterizing fragility to mitigating it\. Fine\-tuning TFMs on metagenomic cohorts would align their priors with the compositional, zero\-inflated structure that synthetic pretraining does not capture, while data augmentation exposing models to support\-query sparsity mismatch during training, together with post\-hoc calibration, offers complementary routes toward greater stability under distribution shift\.

Data representation\.We evaluated models on raw relative abundances, leaving compositional transforms such as the centered log\-ratio \(CLR\) and its robust variants unexplored\[[15](https://arxiv.org/html/2606.24995#bib.bib15)\]\. Whether such transforms help or hinder TFM representations remains an open question: these models apply their own internal normalization, which an external log\-ratio transform might disrupt rather than complement\.

Perturbation refinement\.Zero Imputation fills structural zeros by sampling from each feature’s empirical non\-zero distribution\. However, technical false zeros \(taxa present but undetected owing to limited sequencing depth\) are more plausibly located just above the detection limit; restricting sampling to the lower tail of each feature’s distribution would therefore yield a more biologically faithful densification and constitutes a natural refinement of Algorithm 3\.

Feature selection criterion\.The partition into protected and perturbed features relies on ANOVA F\-tests and a Random Forest wrapper, which may not capture the non\-linear dependencies that foundation models exploit\. A model\-agnostic attribution method such as SHAP, or controlled experiments with a known ground\-truth signal, would help clarify whether the measured degradation reflects genuine robustness gaps or artifacts of the importance criterion\. Such analyses would also help bridge the gap between data\-driven importance and biological relevance, since taxa deemed uninformative by our criterion may nonetheless be clinically meaningful\.

Benchmark extension and biological validation\.The benchmark could also be extended with larger context sizes, alternative support\-query compositions, and a broader model pool — though the Pasolli cohorts, spanning four disease contexts and six datasets of varying size, class balance, and sparsity structure, already provide a representative testbed for the biological variability of interest\. Targeted biological validation — for instance, datasets with experimentally verified absent taxa — would further strengthen the cross\-model evidence reported in[Section˜5](https://arxiv.org/html/2606.24995#S5)and help establish whether the sparsity patterns that TFMs rely on reflect genuine biological signal or cohort\-specific artifacts\.

\{credits\}

#### 7\.0\.1Acknowledgements

This work was supported by a grant from the French Agence Nationale de la Recherche \(ANR\) for the DeepIntegrOmics project \(number ANR\-21\-CE45\-0030\)\. This work was granted access to the HPC resources of IDRIS \(Institut du développement et des ressources en informatique scientifique\) under the allocations 2023\-AD011014580, 2024\-AD011014580R1 and 2024\-AD011015723R1 made by GENCI \(Grand Équipement National de Calcul Intensif\)\.

## References

- \[1\]Breiman, L\.: Random forests\. Machine learning45\(1\), 5–32 \(2001\)
- \[2\]Busato, S\., Gordon, M\., Chaudhari, M\., Jensen, I\., Akyol, T\., Andersen, S\., Williams, C\.: Compositionality, sparsity, spurious heterogeneity, and other data\-driven challenges for machine learning algorithms within plant microbiome studies\. Current Opinion in Plant Biology71, 102326 \(2023\)
- \[3\]Chen, T\., Guestrin, C\.: Xgboost: A scalable tree boosting system\. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining\. pp\. 785–794 \(2016\)
- \[4\]DeLong, E\.R\., DeLong, D\.M\., Clarke\-Pearson, D\.L\.: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach\. Biometrics pp\. 837–845 \(1988\)
- \[5\]Forry, S\.P\., Servetas, S\.L\., Kralj, J\.G\., Soh, K\., Hadjithomas, M\., Cano, R\., Carlin, M\., Amorim, M\.G\.d\., Auch, B\., Bakker, M\.G\., et al\.: Variability and bias in microbiome metagenomic sequencing: an interlaboratory study comparing experimental protocols\. Scientific reports14\(1\), 9785 \(2024\)
- \[6\]Gardner, J\., Perdomo, J\.C\., Schmidt, L\.: Large scale transfer learning for tabular data via language modeling\. Advances in Neural Information Processing Systems37, 45155–45205 \(2024\)
- \[7\]Goodfellow, I\.J\., Shlens, J\., Szegedy, C\.: Explaining and harnessing adversarial examples\. arXiv preprint arXiv:1412\.6572 \(2014\)
- \[8\]Grinsztajn, L\., Flöge, K\., Key, O\., Birkel, F\., Jund, P\., Roof, B\., Jäger, B\., Safaric, D\., Alessi, S\., Hayler, A\., et al\.: Tabpfn\-2\.5: Advancing the state of the art in tabular foundation models\. arXiv preprint arXiv:2511\.08667 \(2025\)
- \[9\]Hendrycks, D\., Gimpel, K\.: A baseline for detecting misclassified and out\-of\-distribution examples in neural networks\. arXiv preprint arXiv:1610\.02136 \(2016\)
- \[10\]Hollmann, N\., Müller, S\., Eggensperger, K\., Hutter, F\.: Tabpfn: A transformer that solves small tabular classification problems in a second\. arXiv preprint arXiv:2207\.01848 \(2022\)
- \[11\]Hollmann, N\., Müller, S\., Purucker, L\., Krishnakumar, A\., Körfer, M\., Hoo, S\.B\., Schirrmeister, R\.T\., Hutter, F\.: Accurate predictions on small data with a tabular foundation model\. Nature637\(8045\), 319–326 \(2025\)
- \[12\]Kolberg, C\., Eggensperger, K\., Pfeifer, N\.: Tabpfn\-wide: Continued pre\-training for extreme feature counts\. arXiv preprint arXiv:2510\.06162 \(2025\)
- \[13\]Kumar, B\., Lorusso, E\., Fosso, B\., Pesole, G\.: A comprehensive overview of microbiome data in the light of machine learning applications: categorization, accessibility, and future directions\. Frontiers in microbiology15, 1343572 \(2024\)
- \[14\]Ma, J\., Thomas, V\., Hosseinzadeh, R\., Labach, A\., Kamkari, H\., Cresswell, J\.C\., Golestan, K\., Yu, G\., Caterini, A\.L\., Volkovs, M\.: Tabdpt: Scaling tabular foundation models on real data\. arXiv preprint arXiv:2410\.18164 \(2024\)
- \[15\]Martino, C\., Morton, J\.T\., Marotz, C\.A\., Thompson, L\.R\., Tripathi, A\., Knight, R\., Zengler, K\.: A novel sparse compositional technique reveals microbial perturbations\. MSystems4\(1\), 10–1128 \(2019\)
- \[16\]Müller, S\., Hollmann, N\., Arango, S\.P\., Grabocka, J\., Hutter, F\.: Transformers can do bayesian inference\. arXiv preprint arXiv:2112\.10510 \(2021\)
- \[17\]Pasolli, E\., Schiffer, L\., Manghi, P\., Renson, A\., Obenchain, V\., Truong, D\.T\., Beghini, F\., Malik, F\., Ramos, M\., Dowd, J\.B\., et al\.: Accessible, curated metagenomic data through experimenthub\. Nature methods14\(11\), 1023–1024 \(2017\)
- \[18\]Perciballi, G\., Granese, F\., Fall, A\., Zehraoui, F\., Prifti, E\., Zucker, J\.D\.: Adapting tabpfn for zero\-inflated metagenomic data\. In: NeurIPS 2024 Third Table Representation Learning Workshop \(2024\)
- \[19\]Qu, J\., HolzmÞller, D\., Varoquaux, G\., Morvan, M\.L\.: Tabicl: A tabular foundation model for in\-context learning on large data\. arXiv preprint arXiv:2502\.05564 \(2025\)
- \[20\]Qu, J\., Holzmüller, D\., Varoquaux, G\., Morvan, M\.L\.: Tabiclv2: A better, faster, scalable, and open tabular foundation model\. arXiv preprint arXiv:2602\.11139 \(2026\)
- \[21\]Shreiner, A\.B\., Kao, J\.Y\., Young, V\.B\.: The gut microbiome in health and in disease\. Current opinion in gastroenterology31\(1\), 69–75 \(2015\)
- \[22\]Spinaci, M\., Polewczyk, M\., Schambach, M\., Thelin, S\.: Contexttab: A semantics\-aware tabular in\-context learner\. arXiv preprint arXiv:2506\.10707 \(2025\)
- \[23\]Vanschoren, J\., Van Rijn, J\.N\., Bischl, B\., Torgo, L\.: Openml: networked science in machine learning\. ACM SIGKDD Explorations Newsletter15\(2\), 49–60 \(2014\)
- \[24\]Zeng, Y\., Dinh, T\., Kang, W\., Mueller, A\.C\.: Tabflex: Scaling tabular learning to millions with linear attention\. arXiv preprint arXiv:2506\.05584 \(2025\)

## Appendix 0\.ASupplementary Material to Section 3

### 0\.A\.1Perturbed data example

![Refer to caption](https://arxiv.org/html/2606.24995v1/images/abundance_WT2D__remove_features__trajectories.png)\(a\)Feature Removal
![Refer to caption](https://arxiv.org/html/2606.24995v1/images/abundance_WT2D__densification__trajectories.png)\(b\)Zero Imputation
![Refer to caption](https://arxiv.org/html/2606.24995v1/images/abundance_WT2D__sparsity__trajectories.png)\(c\)Zero Inflation

Figure 5:Mean abundance distributions of individual features across increasing perturbation levels, stratified by class \(Controls and Cases\)– Red lines indicate ANOVA\-selected protected features, which are preserved during perturbation and show consistent abundance enrichment relative to unprotected features \(grey\)\. Boxplots summarize the distribution of all features at each perturbation level; their parallel trajectories indicate that neither perturbation disrupts the relative class\-level structure\.
### 0\.A\.2Perturbation algorithm pseudocodes

Algorithm 1Feature Removal1:Taxonomic abundance matrix

𝐗∈ℝn×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{n\\times d\}, set of uninformative features

ℱU\\mathcal\{F\}\_\{U\}, number of features to remove

kk\.

2:Taxonomic abundance matrix

𝐗′∈ℝn×\(d−k\)\\mathbf\{X\}^\{\\prime\}\\in\\mathbb\{R\}^\{n\\times\(d\-k\)\}\.

3:

μj←1n​∑i=1nxi​j\\mu\_\{j\}\\leftarrow\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}x\_\{ij\}∀j∈ℱU\\forall\\,j\\in\\mathcal\{F\}\_\{U\}⊳\\trianglerightCompute mean abundance overℱU\\mathcal\{F\}\_\{U\}

4:Sort features

j∈ℱUj\\in\\mathcal\{F\}\_\{U\}in descending order by

μj\\mu\_\{j\}
5:

ℱ←\{sorted​\(ℱU\)\}1:k\\mathcal\{F\}\\leftarrow\\\{\\texttt\{sorted\}\(\\mathcal\{F\}\_\{U\}\)\\\}\_\{1:k\}⊳\\trianglerightSelect thekkhighest\-abundance uninformative features

6:

𝒦←\{1,…,d\}∖ℱ\\mathcal\{K\}\\leftarrow\\\{1,\\dots,d\\\}\\setminus\\mathcal\{F\}⊳\\trianglerightRetain all columns not selected for removal

7:for

i=1i=1to

nndo

8:

si←∑j∈𝒦xi​js\_\{i\}\\leftarrow\\sum\_\{j\\in\\mathcal\{K\}\}x\_\{ij\}
9:if

si\>0s\_\{i\}\>0then

10:

xi​j′←xi​j/six^\{\\prime\}\_\{ij\}\\leftarrow x\_\{ij\}/s\_\{i\}∀j∈𝒦\\forall\\,j\\in\\mathcal\{K\}⊳\\trianglerightRe\-normalize to restore compositionality

11:else

12:

xi​j′←0x^\{\\prime\}\_\{ij\}\\leftarrow 0∀j∈𝒦\\forall\\,j\\in\\mathcal\{K\}
13:endif

14:endfor

15:return

𝐗′\\mathbf\{X\}^\{\\prime\}

Algorithm 2Zero Inflation1:Taxonomic abundance matrix

𝐗∈ℝn×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{n\\times d\}, set of uninformative features

ℱU\\mathcal\{F\}\_\{U\}, target sparsity

ρ%⋆∈\(0,1\)\\rho^\{\\star\}\_\{\\%\}\\in\(0,1\), threshold

τ=10−6\\tau=10^\{\-6\}, number of iterations

TT, search bounds

γmin≥0\\gamma\_\{\\min\}\\geq 0,

γmax\>γmin\\gamma\_\{\\max\}\>\\gamma\_\{\\min\}\.

2:Convention:Indices

i∈\{1,…,n\}i\\in\\\{1,\\dots,n\\\}denote samples,

j∈\{1,…,d\}j\\in\\\{1,\\dots,d\\\}denote features\.

3:Taxonomic abundance matrix

𝐗′∈ℝn×d\\mathbf\{X\}^\{\\prime\}\\in\\mathbb\{R\}^\{n\\times d\}with sparsity less than

ρ%⋆\\rho^\{\\star\}\_\{\\%\}\.

4:

ρ←∑i,j𝟙​\[xi​j=0\]\\rho\\leftarrow\\sum\_\{i,j\}\\mathds\{1\}\[x\_\{ij\}=0\]⊳\\trianglerightCurrent number of zeros in𝐗\\mathbf\{X\}

5:

ρ⋆←⌊ρ%⋆⋅n⋅d⌋\\rho^\{\\star\}\\leftarrow\\lfloor\\rho^\{\\star\}\_\{\\%\}\\cdot n\\cdot d\\rfloor⊳\\trianglerightTarget number of zeros

6:

ϕU←∑i,j𝟙​\[xi​j\>0\]⋅𝟙​\[j∈ℱU\]\\phi\_\{U\}\\leftarrow\\sum\_\{i,j\}\\mathds\{1\}\[x\_\{ij\}\>0\]\\cdot\\mathds\{1\}\[j\\in\\mathcal\{F\}\_\{U\}\]⊳\\trianglerightNon\-zero entries available for sparsification

7:

k←min⁡\{ρ⋆−ρ,ϕU\}k\\leftarrow\\min\\\{\\rho^\{\\star\}\-\\rho,\\,\\phi\_\{U\}\\\}⊳\\trianglerightNumber of zeros to introduce

8:if

k<0k<0then

9:raiseValueError: target sparsity

ρ%⋆\\rho^\{\\star\}\_\{\\%\}is already exceeded by the current sparsity

10:endif

11:if

k=0k=0then

12:return

𝐗\\mathbf\{X\}⊳\\trianglerightNo sparsification needed

13:endif

14:

ρU←∑i,j𝟙​\[xi​j=0\]⋅𝟙​\[j∈ℱU\]\\rho\_\{U\}\\leftarrow\\sum\_\{i,j\}\\mathds\{1\}\[x\_\{ij\}=0\]\\cdot\\mathds\{1\}\[j\\in\\mathcal\{F\}\_\{U\}\]⊳\\trianglerightCurrent zeros inℱU\\mathcal\{F\}\_\{U\}

15:

𝐗∗←𝐗\\mathbf\{X\}^\{\*\}\\leftarrow\\mathbf\{X\},

d∗←∞d^\{\*\}\\leftarrow\\infty
16:for

t=1t=1to

TTdo

17:

γ←\(γmin\+γmax\)/2\\gamma\\leftarrow\(\\gamma\_\{\\min\}\+\\gamma\_\{\\max\}\)/2
18:

x~i​j←xi​j\\tilde\{x\}\_\{ij\}\\leftarrow x\_\{ij\}∀j∉ℱU\\forall\\,j\\notin\\mathcal\{F\}\_\{U\}⊳\\trianglerightInformative features are left unchanged

19:

x~i​j←\(xi​j\)1\+γ⋅𝟙​\[\(xi​j\)1\+γ≥τ\]\\tilde\{x\}\_\{ij\}\\leftarrow\(x\_\{ij\}\)^\{1\+\\gamma\}\\cdot\\mathds\{1\}\[\(x\_\{ij\}\)^\{1\+\\gamma\}\\geq\\tau\]∀j∈ℱU\\forall\\,j\\in\\mathcal\{F\}\_\{U\}⊳\\trianglerightPower transformation with threshold

20:

ρU′←∑i,j𝟙​\[x~i​j=0\]⋅𝟙​\[j∈ℱU\]−ρU\\rho\_\{U\}^\{\\prime\}\\leftarrow\\sum\_\{i,j\}\\mathds\{1\}\[\\tilde\{x\}\_\{ij\}=0\]\\cdot\\mathds\{1\}\[j\\in\\mathcal\{F\}\_\{U\}\]\-\\rho\_\{U\}⊳\\trianglerightNewly introduced zeros

21:if

\|ρU′−k\|<d∗\|\\rho\_\{U\}^\{\\prime\}\-k\|<d^\{\*\}then

22:

d∗←\|ρU′−k\|d^\{\*\}\\leftarrow\|\\rho\_\{U\}^\{\\prime\}\-k\|;

xi​j∗←x~i​j​∀i,jx^\{\*\}\_\{ij\}\\leftarrow\\tilde\{x\}\_\{ij\}\\ \\forall\\,i,j⊳\\trianglerightSave best iterate

23:endif

24:if

\|ρU′−k\|≤1\|\\rho\_\{U\}^\{\\prime\}\-k\|\\leq 1thenbreak

25:endif

26:if

ρU′<k\\rho\_\{U\}^\{\\prime\}<kthen

γmin←γ\\gamma\_\{\\min\}\\leftarrow\\gamma
27:else

γmax←γ\\gamma\_\{\\max\}\\leftarrow\\gamma
28:endif

29:endfor

30:for

i=1i=1to

nndo⊳\\trianglerightRe\-normalize rows to restore compositionality

31:

si←∑j=1dxi​j∗s\_\{i\}\\leftarrow\\sum\_\{j=1\}^\{d\}x^\{\*\}\_\{ij\}
32:if

si\>0s\_\{i\}\>0then

33:

xi​j′←xi​j∗/six^\{\\prime\}\_\{ij\}\\leftarrow x^\{\*\}\_\{ij\}/s\_\{i\}∀j\\forall\\,j
34:else

35:

xi​j′←0x^\{\\prime\}\_\{ij\}\\leftarrow 0∀j\\forall\\,j
36:endif

37:endfor

38:return

𝐗′\\mathbf\{X\}^\{\\prime\}

Algorithm 3Zero Imputation1:Taxonomic abundance matrix

𝐗∈ℝn×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{n\\times d\}, set of uninformative features

ℱU\\mathcal\{F\}\_\{U\}, target sparsity

ρ%⋆∈\(0,1\)\\rho^\{\\star\}\_\{\\%\}\\in\(0,1\)\.

2:Convention:Indices

i∈\{1,…,n\}i\\in\\\{1,\\dots,n\\\}denote samples,

j∈\{1,…,d\}j\\in\\\{1,\\dots,d\\\}denote features

3:Taxonomic abundance matrix

𝐗′∈ℝn×d\\mathbf\{X\}^\{\\prime\}\\in\\mathbb\{R\}^\{n\\times d\}with sparsity less than

ρ%⋆\\rho^\{\\star\}\_\{\\%\}
4:

ρ←∑i,j𝟙​\[xi​j=0\]\\rho\\leftarrow\\sum\_\{i,j\}\\mathds\{1\}\[x\_\{ij\}=0\]⊳\\trianglerightCurrent number of zeros in𝐗\\mathbf\{X\}

5:

ρ⋆←⌊ρ%⋆⋅n⋅d⌋\\rho^\{\\star\}\\leftarrow\\lfloor\\rho^\{\\star\}\_\{\\%\}\\cdot n\\cdot d\\rfloor⊳\\trianglerightTarget number of zeros

6:

k←ρ−ρ⋆k\\leftarrow\\rho\-\\rho^\{\\star\}⊳\\trianglerightNumber of zeros to fill

7:if

k<0k<0then

8:raiseValueError: target sparsity

ρ%⋆\\rho^\{\\star\}\_\{\\%\}is already below the current sparsity

9:endif

10:if

k=0k=0then

11:return

𝐗\\mathbf\{X\}⊳\\trianglerightNo densification needed

12:endif

13:

𝒵←\{\(i,j\):xi​j=0,j∈ℱU\}\\mathcal\{Z\}\\leftarrow\\\{\(i,j\):x\_\{ij\}=0,\\;j\\in\\mathcal\{F\}\_\{U\}\\\}⊳\\trianglerightZero entries available for densification

14:

k←min⁡\{k,\|𝒵\|\}k\\leftarrow\\min\\\{k,\\,\|\\mathcal\{Z\}\|\\\}⊳\\trianglerightCap at available zero entries inℱU\\mathcal\{F\}\_\{U\}

15:for

j∈ℱUj\\in\\mathcal\{F\}\_\{U\}do⊳\\trianglerightCollect observed non\-zero values per feature

16:

𝒱j←\{xi​j:xi​j\>0\}\\mathcal\{V\}\_\{j\}\\leftarrow\\\{x\_\{ij\}:x\_\{ij\}\>0\\\}⊳\\trianglerightNon\-zero values of featurejj

17:endfor

18:if

∀j∈ℱU:\|𝒱j\|=0\\forall\\,j\\in\\mathcal\{F\}\_\{U\}:\\,\|\\mathcal\{V\}\_\{j\}\|=0then

19:return

𝐗\\mathbf\{X\}⊳\\trianglerightNo non\-zero values available to sample from

20:endif

21:

𝒮←\\mathcal\{S\}\\leftarrowsample

kkpositions from

𝒵\\mathcal\{Z\}uniformly at random

22:

𝐗⋆←𝐗\\mathbf\{X\}^\{\\star\}\\leftarrow\\mathbf\{X\}
23:for

ℓ=1\\ell=1to

kkdo

24:

\(i,j\)←𝒮ℓ\(i,j\)\\leftarrow\\mathcal\{S\}\_\{\\ell\}
25:if

\|𝒱j\|\>0\|\\mathcal\{V\}\_\{j\}\|\>0then

26:

xi​j⋆←x^\{\\star\}\_\{ij\}\\leftarrowsample uniformly at random from

𝒱j\\mathcal\{V\}\_\{j\}⊳\\trianglerightDraw from observed non\-zero values of featurejj

27:else

28:

j⋆←j^\{\\star\}\\leftarrowsample uniformly from

\{j⋆⋆∈ℱU:\|𝒱j⋆⋆\|\>0\}\\\{j^\{\\star\\star\}\\in\\mathcal\{F\}\_\{U\}:\|\\mathcal\{V\}\_\{j^\{\\star\\star\}\}\|\>0\\\}
29:

xi​j⋆←x^\{\\star\}\_\{ij\}\\leftarrowsample uniformly at random from

𝒱j⋆\\mathcal\{V\}\_\{j^\{\\star\}\}⊳\\trianglerightFallback: draw from observed non\-zero values of featurej⋆j^\{\\star\}

30:endif

31:endfor

32:for

i=1i=1to

nndo⊳\\trianglerightRe\-normalize rows to restore compositionality

33:

si←∑j=1dxi​j⋆s\_\{i\}\\leftarrow\\sum\_\{j=1\}^\{d\}x^\{\\star\}\_\{ij\}
34:if

si\>0s\_\{i\}\>0then

35:

xi​j′←xi​j⋆/six^\{\\prime\}\_\{ij\}\\leftarrow x^\{\\star\}\_\{ij\}/s\_\{i\}∀j\\forall\\,j
36:else

37:

xi​j′←0x^\{\\prime\}\_\{ij\}\\leftarrow 0∀j\\forall\\,j
38:endif

39:endfor

40:return

𝐗′\\mathbf\{X\}^\{\\prime\}

## Appendix 0\.BSupplementary Material to Section 4\.1

#### 0\.B\.0\.1Metrics

Performance is measured by AUROC, where the effect of each perturbation is given asΔ​AUROC=AUROCperturbed−AUROCbaseline\\Delta\\text\{AUROC\}=\\text\{AUROC\}\_\{\\text\{perturbed\}\}\-\\text\{AUROC\}\_\{\\text\{baseline\}\}\. Statistical significance of performance degradation is assessed using the DeLong test\[[4](https://arxiv.org/html/2606.24995#bib.bib4)\], applied directly to the aggregated predicted probabilities across all 5 folds\. This yields approximatelynnpaired samples per test, one per dataset sample, providing substantially more statistical power than a fold\-level test\. Significance is assessed atα=0\.05\\alpha=0\.05\.

## Appendix 0\.CSupplementary Material to Section 5

### 0\.C\.1Informative vs\. uninformative feature selection

![Refer to caption](https://arxiv.org/html/2606.24995v1/images/protected_features.png)

Figure 6:AUROC vs top\-k features removed– Features are ranked by ANOVA F\-score, and a Random Forest is evaluated via 5\-fold CV as top features are iteratively removed\. The number of protected features is set just before AUROC drops by 3%\.

Table 1:Number of informative selected features per dataset–\|ℱI\|\|\\mathcal\{F\}\_\{I\}\|\.
### 0\.C\.2Semantic richness \- robustness tradeoff

Table 2:Baseline AUROC \(mean±\\pmstd\) across 5\-fold CV of SOTA ML and TFM on Pasolli dataset\[[17](https://arxiv.org/html/2606.24995#bib.bib17)\]\.The concordance analysis, together with the per\-perturbation results, points to a broader pattern\.TabICLandContextTabco\-degrade under zero inflation despite having different architectures\. What they share is that both have learned realistic abundance distributions during pre\-training\. InTabICL, the column\-wise Set Transformer builds per\-feature marginals that feed into CLS\-based row embeddings\[[19](https://arxiv.org/html/2606.24995#bib.bib19)\], zeroing out values collapses these marginals\. InContextTab, column\-name embeddings tie each feature to a learned lexical representation of the corresponding taxon name; this semantic grounding becomes uninformative when the abundance values it depends on are replaced by zeros\.

This points to a tradeoff between semantic richness and robustness\. Models that have learned how to represent a feature, whether through distributional statistics or learned column embeddings, are also more sensitive to how that feature is corrupted\.

TabPFN, trained on synthetic data without domain\-specific priors, largely avoids this vulnerability \(nsig=3n\_\{\\text\{sig\}\}=3under zero inflation\)\. For practitioners, this means that the choice between semantically rich and distribution\-agnostic models should depend on the expected data quality at deployment\. When sparsity and compositional structure are stable across training and test environments, semantic models can leverage their richer representations\. When these properties vary, as is common in multi\-site microbiome studies with different sequencing protocols, a model with fewer distributional assumptions may be the safer choice\.