TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification

# TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
Source: [https://arxiv.org/html/2607.27289](https://arxiv.org/html/2607.27289)
Yu Chang\\equalcontrib1, Anzhe Cheng\\equalcontrib1, Chenwei Wu\\equalcontrib2, Zhuoran Wang3, Jiahao Chen4, Tamoghna Chattopadhyay1, Sophia I\. Thomopoulos1, Paul M\. Thompson1, Liyue Shen2, Paul Bogdan\\corresponding1

###### Abstract

The promise of multimodal fusion lies in combining complementary sources of evidence, yet more evidence does not always yield a better prediction\. Recent multimodal models have advanced fusion through richer cross\-modal interaction and sample\-adaptive fusion\. However, the influence assigned to a modality during fusion does not reveal whether that source is unreliable, redundant, or poorly matched to a specialized expert\. To address this limitation, we introduceTIER\-MoE, a risk\-guided subspace mixture\-of\-experts model that defines sample\-specific modality reliability as the prediction loss its unimodal predictor is expected to incur\. This risk is learned from out\-of\-fold predictions generated by models that were not trained on the corresponding sample\. TIER\-MoE combines the estimated risk with expert\-specific subspace compatibility for sparse modality–expert routing, while an always\-active shared path preserves multimodal complementarity\. We evaluate TIER\-MoE on four public multimodal biomedical datasets spanning Alzheimer’s disease status, skin\-lesion malignancy, and retinal classification\. Results demonstrate its superiority over state\-of\-the\-art methods in predictive performance and probability calibration, with consistent improvements in Macro\-F1 and Brier score and strong zero\-shot generalization to an external cohort\.

## Introduction

Multimodal biomedical classification is widely used for disease diagnosis, prognosis, and patient stratification\(Doanet al\.[2026](https://arxiv.org/html/2607.27289#bib.bib21)\)\. As many diseases affect several biological systems simultaneously, a single measurement often provides only a partial view of the underlying condition\. Multimodal biomedical prediction addresses this limitation by combining observations of anatomy, tissue microstructure, physiology, and clinical state\(Acostaet al\.[2022](https://arxiv.org/html/2607.27289#bib.bib24); Stahlschmidtet al\.[2022](https://arxiv.org/html/2607.27289#bib.bib16)\)\. For example, T1\-weighted \(T1w\) MRI captures macroscopic brain anatomy, diffusion tensor imaging \(DTI\) reflects white\-matter microstructure\(Lorioet al\.[2016](https://arxiv.org/html/2607.27289#bib.bib22); Taeet al\.[2018](https://arxiv.org/html/2607.27289#bib.bib23)\), and structured clinical metadata provide complementary information about patient state and disease presentation\(Acostaet al\.[2022](https://arxiv.org/html/2607.27289#bib.bib24)\)\. In principle, these modalities should improve prediction by providing complementary evidence\. However, in practice, their quality and predictive value vary across subjects\(Hanet al\.[2022](https://arxiv.org/html/2607.27289#bib.bib25); Zhanget al\.[2023](https://arxiv.org/html/2607.27289#bib.bib33)\), and multimodal models can even perform worse than their strongest unimodal components in some cases\(Huanget al\.[2022](https://arxiv.org/html/2607.27289#bib.bib34)\)\. The main difficulty is therefore not simply how to combine multiple representations, but how to determine which evidence is reliable for each subject without discarding information that remains useful through cross\-modal interaction\.

![Refer to caption](https://arxiv.org/html/2607.27289v1/motivation.png)Figure 1:Motivation for subject\-adaptive fusion\. \(a\) The modality with lower subject\-level OOF Brier loss varies across subjects\. \(b\) Standard fusion baselines remain below T1w across repeated runs\. \(c\) DTI is uniquely correct for 39 of 367 subjects, demonstrating nonredundant evidence\.Existing fusion methods do not fully resolve this reliability–complementarity trade\-off\(Hanet al\.[2022](https://arxiv.org/html/2607.27289#bib.bib25); Neverovaet al\.[2016](https://arxiv.org/html/2607.27289#bib.bib26); Wanget al\.[2020](https://arxiv.org/html/2607.27289#bib.bib27)\)\. Jointly trained fusion architectures include gated models such as GMU\(Arevaloet al\.[2017](https://arxiv.org/html/2607.27289#bib.bib29)\)and attention\-based models such as MBT\(Nagraniet al\.[2021](https://arxiv.org/html/2607.27289#bib.bib30)\)\. Under joint optimization, these models can over\-rely on the globally dominant modality, inducing*modality competition*that underutilizes complementary evidence from weaker modalities and may leave fusion below the best unimodal predictor\(Huanget al\.[2022](https://arxiv.org/html/2607.27289#bib.bib34); Wanget al\.[2020](https://arxiv.org/html/2607.27289#bib.bib27)\)\. Adaptive fusion methods adjust modality influence per sample using evidential uncertainty in TMC\(Hanet al\.[2023](https://arxiv.org/html/2607.27289#bib.bib14)\), quality\-aware weighting in QMF\(Zhanget al\.[2023](https://arxiv.org/html/2607.27289#bib.bib33)\), collaborative confidence in PDF\(Caoet al\.[2024](https://arxiv.org/html/2607.27289#bib.bib15)\), or probabilistic credibility in Cred\-MF\(Sidheekhet al\.[2025](https://arxiv.org/html/2607.27289#bib.bib3)\)\. Although these methods adapt modality influence across samples, their scores represent uncertainty, quality, confidence, or credibility rather than directly predicting the loss that a modality\-specific classifier is expected to incur on the current sample\. More importantly, reliability is usually used directly to control fusion: once a modality is judged less reliable, its contribution is reduced correspondingly\. This treatment conflates two distinct cases: a modality that is unreliable for the current subject and one that is weak in isolation but complementary in fusion\. Our ADNI analysis illustrates this distinction\. As shown in Fig\.[1](https://arxiv.org/html/2607.27289#Sx1.F1), the more reliable modality, as indicated by lower subject\-level OOF Brier loss, varies across subjects \(a\), yet conventional T1w–DTI fusion remains below the T1w predictor across repeated runs \(b\)\. Although DTI is weaker in aggregate, it uniquely produces the correct prediction for 39 of 367 subjects, demonstrating nonredundant complementary evidence \(c\)\. Thus, global modality strength does not describe subject\-level reliability, and subject\-level reliability alone does not measure the complementary value of a modality\.

![Refer to caption](https://arxiv.org/html/2607.27289v1/Figure2.png)Figure 2:Overview ofTIER\-MoE\. \(a\) Calibrated unimodal predictions form a risk\-weighted reliability branch, while shared context and risk\-guided sparse experts form an interaction branch; their predictions are mixed byγ\\gamma\. \(b\) The risk estimator is trained on grouped OOF evidence and realized Brier\-loss targets, then frozen\. \(c\) Positive risk\-adjusted subspace\-affinity scores select Top\-KKmodality–expert routes whose weighted residuals augment the shared path\.Beyond modality reliability, biomedical image classification is further complicated by heterogeneity in disease\-relevant patterns\. Patients sharing the same diagnostic label can exhibit distinct atrophy trajectories, while structural MRI and DTI emphasize different disease\-relevant regions and provide complementary diagnostic information\(Poulakiset al\.[2022](https://arxiv.org/html/2607.27289#bib.bib31); Bronet al\.[2017](https://arxiv.org/html/2607.27289#bib.bib32)\)\. This heterogeneity suggests that structural alterations may dominate for some subjects, whereas diffusion\-related changes or their interaction with structural features may be more informative for others\. A single fusion function is unlikely to represent all such patterns equally well\. Mixture\-of\-experts \(MoE\) models provide conditional computation through specialized experts that are selectively activated for each input\(Shazeeret al\.[2017](https://arxiv.org/html/2607.27289#bib.bib35); Riquelmeet al\.[2021](https://arxiv.org/html/2607.27289#bib.bib17)\)\. Recent multimodal MoE models instantiate expert specialization in different ways: M4oE combines modality\-specific and shared modality–task experts\(Wuet al\.[2025](https://arxiv.org/html/2607.27289#bib.bib18)\), Flex\-MoE routes different observed modality combinations\(Yunet al\.[2024](https://arxiv.org/html/2607.27289#bib.bib19)\), and I2MoE models distinct cross\-modal interactions with specialized experts\(Xinet al\.[2025](https://arxiv.org/html/2607.27289#bib.bib20)\)\. ERMoE further aligns expert selection with learned representation subspaces\(Chenget al\.[2026](https://arxiv.org/html/2607.27289#bib.bib12)\)\. However, these methods do not jointly couple task\-grounded, subject\-specific modality risk with expert compatibility\. Thus, modality reliability, cross\-modal complementarity, and expert specialization remain uncoupled in existing routing formulations\.

To address these limitations, we propose the*Trust\-Informed Expert Routing via Conditional Modality Risk*\(TIER\-MoE\), a hierarchical framework that connects subject\-specific modality reliability estimation with expert specialization for multimodal fusion\.TIER\-MoEfirst estimates the conditional risk of each modality, defined as the expected prediction loss of its unimodal predictor for the current subject\. For each subject, the risk target is the loss of an out\-of\-fold unimodal prediction generated by a model trained on the remaining subjects\. These estimates form a reliability anchor that preserves dependable unimodal evidence without acting as the sole fusion mechanism\. In parallel,TIER\-MoElearns low\-rank expert subspaces and measures their affinity with each modality representation\. A joint risk–affinity criterion then selects sparse modality–expert routes, assigning reliable evidence to experts whose subspaces are suitable for processing it\. A shared interaction path retains information across all available modalities, while routed expert residuals capture specialized subject\-dependent interactions\. This design assigns distinct roles to three quantities that prior methods either conflate or model separately: the task risk of each modality\-specific predictor, its complementary value during fusion, and its compatibility with a specialized expert\.

The primary contributions of this work are summarized as follows:

- •We formulate subject\-specific modality reliability as the conditional expected prediction loss of a unimodal predictor and learn it from subject\-level out\-of\-fold predictions\. This definition provides a common, task\-grounded reliability measure that can be evaluated directly against held\-out prediction errors\.
- •We introduceTIER\-MoE, which combines conditional modality risk with expert\-subspace affinity for subject\-dependent modality–expert routing\. Its reliability anchor and shared interaction path preserve reliable unimodal predictions while retaining complementary evidence from modalities that are weaker in isolation\.
- •We evaluateTIER\-MoEacross four biomedical cohorts and three classification tasks under standard prediction, modality degradation, missingness, cross\-modal conflict, and external cohort shift, demonstrating consistent improvements in prediction, calibration, robustness, and harmful fusion reduction\.

## Method

### Network Architecture

Figure[2](https://arxiv.org/html/2607.27289#Sx1.F2)illustrates the proposedTIER\-MoE, short for*Trust\-Informed Expert Routing via Conditional Modality Risk*\. The framework assigns a distinct mechanism to each quantity identified in the introduction: conditional risk estimates the expected loss of each modality\-specific predictor, an always\-active shared path maintains cross\-modal interaction independently of sparse routing, and expert\-subspace affinity measures representation\-level compatibility\. The reliability branch forms a risk\-weighted unimodal prediction, whereas the interaction branch adds sparsely routed expert residuals to the shared representation\. Their predictions are combined using a validation\-locked coefficient\.

### Cross\-Fitted Conditional Modality Risk

We define sample\-specific modality reliability through*conditional modality risk*, the expected prediction loss of the corresponding unimodal predictor:

Rm​\(Vi\)=𝔼​\[Li,m∣Vi\]\.R\_\{m\}\(V\_\{i\}\)=\\mathbb\{E\}\\\!\\left\[L\_\{i,m\}\\mid V\_\{i\}\\right\]\.\(1\)whereVi=\{\(vi,m,ai,m\)\}m=1MV\_\{i\}=\\\{\(v\_\{i,m\},a\_\{i,m\}\)\\\}\_\{m=1\}^\{M\}collects inference\-time evidence across modalities\. The local vectorvi,mv\_\{i,m\}contains the calibrated predictive distribution, predictive uncertainty, technical quality indicators, cross\-modal disagreement, and fold\-comparable representation summaries, whileai,ma\_\{i,m\}indicates modality availability\.

#### Label\-honest risk targets\.

The conditional expectation in Eq\. \([1](https://arxiv.org/html/2607.27289#Sx2.E1)\) is not directly observed\. We construct its realized training targets from out\-of\-fold \(OOF\) unimodal predictions\. For multiclass classification, we use the normalized Brier loss:

Li,mOOF=12​‖pi,mOOF−eYi‖22,L\_\{i,m\}^\{\\mathrm\{OOF\}\}=\\frac\{1\}\{2\}\\left\\\|p\_\{i,m\}^\{\\mathrm\{OOF\}\}\-e\_\{Y\_\{i\}\}\\right\\\|\_\{2\}^\{2\},\(2\)whereeYie\_\{Y\_\{i\}\}is the one\-hot label vector andpi,mOOFp\_\{i,m\}^\{\\mathrm\{OOF\}\}is the calibrated prediction of a unimodal model trained without subjectii\. The normalization givesLi,mOOF∈\[0,1\]L\_\{i,m\}^\{\\mathrm\{OOF\}\}\\in\[0,1\]\.

We obtain these targets through grouped subject\-level cross\-fitting\. For each fold, the complete supervised unimodal branch is trained on the remaining subjects\. Its probability calibrator is fitted using only data within that fold’s training partition\. The calibrated model is then evaluated on the held\-out subjects to obtainpi,mOOFp\_\{i,m\}^\{\\mathrm\{OOF\}\}\. Thus, the label of subjectiiis used to evaluate its realized loss but never to fit the predictor or calibrator that produces its OOF prediction\.

#### Risk estimator\.

A shared mask\-aware estimator embeds each modality’s local evidence and conditions its risk on the pooled context of all available modalities:

ui,m\\displaystyle u\_\{i,m\}=ϕθ​\(\[vi,m,em\]\),\\displaystyle=\\phi\_\{\\theta\}\(\[v\_\{i,m\},e\_\{m\}\]\),\(3\)ri,mraw\\displaystyle r\_\{i,m\}^\{\\mathrm\{raw\}\}=σ​\(hθ​\(\[ui,m,∑j=1Mai,j​ξθ​\(ui,j\)∑j=1Mai,j\]\)\),\\displaystyle=\\sigma\\\!\\left\(h\_\{\\theta\}\\\!\\left\(\\left\[u\_\{i,m\},\\frac\{\\sum\_\{j=1\}^\{M\}a\_\{i,j\}\\xi\_\{\\theta\}\(u\_\{i,j\}\)\}\{\\sum\_\{j=1\}^\{M\}a\_\{i,j\}\}\\right\]\\right\)\\right\),whereeme\_\{m\}is a learned modality embedding andϕθ\\phi\_\{\\theta\},ξθ\\xi\_\{\\theta\}, andhθh\_\{\\theta\}are shared nonlinear mappings\.

The estimator is supervised by the realized OOF Brier losses:

ℒrisk=∑i=1N∑m=1Mai,m​\(ri,mraw−Li,mOOF\)2∑i=1N∑m=1Mai,m\.\\mathcal\{L\}\_\{\\mathrm\{risk\}\}=\\frac\{\\displaystyle\\sum\_\{i=1\}^\{N\}\\sum\_\{m=1\}^\{M\}a\_\{i,m\}\\left\(r\_\{i,m\}^\{\\mathrm\{raw\}\}\-L\_\{i,m\}^\{\\mathrm\{OOF\}\}\\right\)^\{2\}\}\{\\displaystyle\\sum\_\{i=1\}^\{N\}\\sum\_\{m=1\}^\{M\}a\_\{i,m\}\}\.\(4\)Squared\-loss regression targets the conditional expectation in Eq\. \(1\)\. We train the risk estimator on all subject\-level OOF evidence–loss pairs and freeze it before optimizing the interaction branch\. For interaction training, we store the OOF\-derived risk estimates

Ri,mOOF=gθ​\(ViOOF,m\),R^\{\\mathrm\{OOF\}\}\_\{i,m\}=g\_\{\\theta\}\(V^\{\\mathrm\{OOF\}\}\_\{i\},m\),wheregθg\_\{\\theta\}denotes the estimator in Eq\. \(3\)\.

#### Risk recalibration\.

After refitting the unimodal predictors on the complete training partition and fitting their probability calibrators on the validation set, the frozen risk estimator producesri,mrawr^\{\\mathrm\{raw\}\}\_\{i,m\}from the final\-model evidence\. To correct the OOF\-to\-refit shift, we fit a low\-capacity monotone map on the validation set:

R^i,m\\displaystyle\\widehat\{R\}\_\{i,m\}=Cm​\(ri,mraw\),\\displaystyle=C\_\{m\}\\\!\\left\(r^\{\\mathrm\{raw\}\}\_\{i,m\}\\right\),\(5\)Cm​\(r\)\\displaystyle C\_\{m\}\(r\)=σ​\(a​logit⁡\(r\)\+bm\),a\>0\.\\displaystyle=\\sigma\\\!\\left\(a\\,\\operatorname\{logit\}\(r\)\+b\_\{m\}\\right\),\\qquad a\>0\.The target is the realized normalized Brier loss of the probability\-calibrated refit unimodal predictor\. The map is frozen for test and external evaluation\.

#### Reliability anchor\.

The calibrated risks define an absolute reliability score and a relative allocation over the available modalities:

ti,m\\displaystyle t\_\{i,m\}=ai,m​exp⁡\(−β​R^i,m\),\\displaystyle=a\_\{i,m\}\\exp\(\-\\beta\\widehat\{R\}\_\{i,m\}\),\(6\)αi,m\\displaystyle\\alpha\_\{i,m\}=πm​ti,m∑j=1Mπj​ti,j,\\displaystyle=\\frac\{\\pi\_\{m\}t\_\{i,m\}\}\{\\sum\_\{j=1\}^\{M\}\\pi\_\{j\}t\_\{i,j\}\},pirel\\displaystyle p\_\{i\}^\{\\mathrm\{rel\}\}=∑m=1Mαi,m​pi,m,\\displaystyle=\\sum\_\{m=1\}^\{M\}\\alpha\_\{i,m\}p\_\{i,m\},whereβ\>0\\beta\>0controls allocation concentration andπm\>0\\pi\_\{m\}\>0is a fixed modality prior, chosen uniformly in all experiments\. Here,ti,mt\_\{i,m\}is an absolute reliability score, whereasαi,m\\alpha\_\{i,m\}is a relative allocation among available modalities\.

### Risk\-Guided Subspace Expert Routing

Letz~i,m\\tilde\{z\}\_\{i,m\}denote modalitymmprojected into the common latent space and augmented with its modality embedding, and letzCLSz\_\{\\mathrm\{CLS\}\}be a learnable shared token\. Availability\-masked self\-attention produces contextualized modality and shared tokens:

\[zi,CLS,zi,1,…,zi,M\]\\displaystyle\\bigl\[z\_\{i,\\mathrm\{CLS\}\},z\_\{i,1\},\\ldots,z\_\{i,M\}\\bigr\]=MaskedAttn⁡\(\[zCLS,z~i,1,…,z~i,M\];𝐚i\)\.\\displaystyle\\quad=\\operatorname\{MaskedAttn\}\\\!\\left\(\\bigl\[z\_\{\\mathrm\{CLS\}\},\\tilde\{z\}\_\{i,1\},\\ldots,\\tilde\{z\}\_\{i,M\}\\bigr\];\\mathbf\{a\}\_\{i\}\\right\)\.LetAi,m​jA\_\{i,mj\}denote the resulting attention weight from modalitymmto an available modalityjj\. We define

ci,m\\displaystyle c\_\{i,m\}=∑j:ai,j=1Ai,m​j​zi,j,\\displaystyle=\\sum\_\{j:\\,a\_\{i,j\}=1\}A\_\{i,mj\}z\_\{i,j\},hishared\\displaystyle h\_\{i\}^\{\\mathrm\{shared\}\}=Hsh​\(zi,CLS\)\.\\displaystyle=H\_\{\\mathrm\{sh\}\}\\\!\\left\(z\_\{i,\\mathrm\{CLS\}\}\\right\)\.The shared representation is computed from all available modalities independently of risk weighting and sparse route selection, maintaining an always\-active path for cross\-modal interaction\.

In parallel with the shared path, TIER\-MoE maintains a bank ofEEsparse residual experts\. For routing, each experte∈\{1,…,E\}e\\in\\\{1,\\ldots,E\\\}is associated with a low\-rank subspaceBe∈ℝd×rB\_\{e\}\\in\\mathbb\{R\}^\{d\\times r\}, wherer<dr<d\. Its columns are encouraged to be orthonormal during training\. Inspired by representation\-aligned expert routing, ERMoE\(Chenget al\.[2026](https://arxiv.org/html/2607.27289#bib.bib12)\), we define modality–expert compatibility as

ψi,m,e=z¯i,m⊤​Be​Be⊤​c¯i,m,\\psi\_\{i,m,e\}=\\bar\{z\}\_\{i,m\}^\{\\top\}B\_\{e\}B\_\{e\}^\{\\top\}\\bar\{c\}\_\{i,m\},\(7\)wherez¯i,m\\bar\{z\}\_\{i,m\}andc¯i,m\\bar\{c\}\_\{i,m\}areℓ2\\ell\_\{2\}\-normalized before projection\. The compatibility score approaches zero when either the modality token or its context has little support in the expert subspace\. It therefore expresses expert compatibility rather than modality reliability\. During interaction\-branch training, Eq\. \([8](https://arxiv.org/html/2607.27289#Sx2.E8)\) uses the stored OOF\-derived estimateRi,mOOFR^\{\\mathrm\{OOF\}\}\_\{i,m\}\. Validation, test, and external evaluation use the recalibrated final\-model riskR^i,m\\widehat\{R\}\_\{i,m\}from Eq\. \([5](https://arxiv.org/html/2607.27289#Sx2.E5)\)\.

Risk and compatibility are combined at the modality–expert\-pair level:

Gi,m,e=ψi,m,e−η​R^i,m\.G\_\{i,m,e\}=\\psi\_\{i,m,e\}\-\\eta\\widehat\{R\}\_\{i,m\}\.\(8\)whereη≥0\\eta\\geq 0controls the risk penalty\. We consider only available modality–expert pairs with positive scores and retain at mostKrK\_\{r\}routes:

𝒞i\\displaystyle\\mathcal\{C\}\_\{i\}=\{\(m,e\)∣ai,m=1,Gi,m,e\>0\},\\displaystyle=\\left\\\{\(m,e\)\\mid a\_\{i,m\}=1,\\;G\_\{i,m,e\}\>0\\right\\\},\(9\)𝒮i\\displaystyle\\mathcal\{S\}\_\{i\}=TopKKr⁡\(𝒞i;Gi\)\.\\displaystyle=\\operatorname\{TopK\}\_\{K\_\{r\}\}\\\!\\left\(\\mathcal\{C\}\_\{i\};G\_\{i\}\\right\)\.
The compatibility term rewards expert\-subspace alignment, whereas conditional risk penalizes evidence likely to incur prediction loss\. Thus, a moderately risky modality may remain routable when strongly aligned with an expert, while low risk alone does not force assignment to an unsuitable expert\. Cross\-modal interaction remains available through the shared path\.

### Interaction Prediction and Training

For a nonempty selected set𝒮i\\mathcal\{S\}\_\{i\}, the risk\-adjusted routing scores are normalized over the activated modality–expert pairs:

ρi,m,e=exp⁡\(Gi,m,e/τ\)∑\(j,e′\)∈𝒮iexp⁡\(Gi,j,e′/τ\),\(m,e\)∈𝒮i,\\rho\_\{i,m,e\}=\\frac\{\\exp\(G\_\{i,m,e\}/\\tau\)\}\{\\displaystyle\\sum\_\{\(j,e^\{\\prime\}\)\\in\\mathcal\{S\}\_\{i\}\}\\exp\(G\_\{i,j,e^\{\\prime\}\}/\\tau\)\},\\qquad\(m,e\)\\in\\mathcal\{S\}\_\{i\},\(10\)whereτ\>0\\tau\>0controls the concentration of the routing distribution\. A smallerτ\\tauplaces more weight on the highest\-scoring routes, whereas a largerτ\\tauproduces a softer allocation among the selected pairs\.

Each selected expert processes its assigned modality representation through a low\-rank residual transformation:

re​\(z\)=Ue​ϕ​\(De​Be⊤​z\),r\_\{e\}\(z\)=U\_\{e\}\\phi\\\!\\left\(D\_\{e\}B\_\{e\}^\{\\top\}z\\right\),\(11\)whereBeB\_\{e\}projects the input into expertee’s routing subspace,DeD\_\{e\}is a learned diagonal transformation,UeU\_\{e\}maps the transformed features back to the shared representation space, andϕ\\phiis a nonlinear activation\.

The selected expert outputs first form a sparse, sample\-specific correction\. This correction is then added to the always\-active shared representation, and the resulting interaction representation is mapped to a predictive distribution:

hisparse\\displaystyle h\_\{i\}^\{\\mathrm\{sparse\}\}=∑\(m,e\)∈𝒮iρi,m,e​re​\(zi,m\),\\displaystyle=\\sum\_\{\(m,e\)\\in\\mathcal\{S\}\_\{i\}\}\\rho\_\{i,m,e\}\\,r\_\{e\}\(z\_\{i,m\}\),\(12\)hiint\\displaystyle h\_\{i\}^\{\\mathrm\{int\}\}=hishared\+hisparse,\\displaystyle=h\_\{i\}^\{\\mathrm\{shared\}\}\+h\_\{i\}^\{\\mathrm\{sparse\}\},piint\\displaystyle p\_\{i\}^\{\\mathrm\{int\}\}=softmax⁡\(Hint​\(hiint\)\)\.\\displaystyle=\\operatorname\{softmax\}\\\!\\left\(H\_\{\\mathrm\{int\}\}\(h\_\{i\}^\{\\mathrm\{int\}\}\)\\right\)\.Here,hisparseh\_\{i\}^\{\\mathrm\{sparse\}\}contains only the selected expert residuals\. Becausehisharedh\_\{i\}^\{\\mathrm\{shared\}\}is computed independently of𝒮i\\mathcal\{S\}\_\{i\}, an empty route set yieldshiint=hisharedh\_\{i\}^\{\\mathrm\{int\}\}=h\_\{i\}^\{\\mathrm\{shared\}\}\. Null routing therefore suppresses sparse expert computation without removing the shared cross\-modal path, and only selected experts are evaluated\.

To balance reliability\-guided unimodal evidence with cross\-modal interaction, we combine the two branch predictions:

pifinal=\(1−γ\)​pirel\+γ​piint,p\_\{i\}^\{\\mathrm\{final\}\}=\(1\-\\gamma\)p\_\{i\}^\{\\mathrm\{rel\}\}\+\\gamma p\_\{i\}^\{\\mathrm\{int\}\},\(13\)whereγ∈\[0,1\]\\gamma\\in\[0,1\]is selected on the validation set and fixed for test and external evaluation\.

#### Interaction objective\.

Following representation\-aligned expert parameterization\(Chenget al\.[2026](https://arxiv.org/html/2607.27289#bib.bib12)\), we optimize the interaction branch with the task objective and two complementary subspace regularizers:

ℒint=ℒtask\+λorth​ℒorth\+λsep​ℒsep\.\\mathcal\{L\}\_\{\\mathrm\{int\}\}=\\mathcal\{L\}\_\{\\mathrm\{task\}\}\+\\lambda\_\{\\mathrm\{orth\}\}\\mathcal\{L\}\_\{\\mathrm\{orth\}\}\+\\lambda\_\{\\mathrm\{sep\}\}\\mathcal\{L\}\_\{\\mathrm\{sep\}\}\.\(14\)Here,ℒorth\\mathcal\{L\}\_\{\\mathrm\{orth\}\}promotes semi\-orthogonal input and output bases within each expert, whileℒsep\\mathcal\{L\}\_\{\\mathrm\{sep\}\}encourages distinct routing subspaces across experts\.

#### Training protocol\.

We first generate subject\-level OOF unimodal predictions, losses, and risk evidence within the training partition\. The risk estimator is trained on all OOF evidence–loss pairs and then frozen\. We next refit the unimodal predictors on the complete training partition, fit their probability calibrators and the final risk mapCmC\_\{m\}on validation, and freeze them\. With the final encoders and risk estimator fixed, the interaction branch is trained using the stored OOF\-derived risk estimates\. Finally, both prediction branches are frozen andγ\\gammais selected on validation for locked test and external evaluation\.

## Experiments

Table 1:Overall predictive and probabilistic performance across the three in\-domain biomedical datasets\. AUROC is macro one\-vs\-rest for ADNI and FPRM and binary for PAD\-UFES\-20; F1 denotes Macro\-F1\. In all of the following tables, thebolddenotes the best result andunderlinethe second best result\.We evaluateTIER\-MoEin terms of \(1\) in\-domain prediction and probability quality, \(2\) conditional\-risk validity, \(3\) robustness to unreliable evidence, \(4\) zero\-shot cross\-cohort transfer, and \(5\) component necessity\.

### Experimental Setup

#### Datasets and splits\.

We evaluate three in\-domain modality regimes: T1w MRI and DTI for three\-class AD/MCI/CN classification on ADNI\(Petersenet al\.[2010](https://arxiv.org/html/2607.27289#bib.bib38)\); lesion images and clinical metadata for binary malignancy classification on PAD\-UFES\-20\(Pachecoet al\.[2020](https://arxiv.org/html/2607.27289#bib.bib39)\); and fundus photographs and retinal blood\-flow videos for three\-class retinal classification on FPRM\(Zhanget al\.[2024](https://arxiv.org/html/2607.27289#bib.bib40)\)\. We use fixed group\-disjoint train/validation/test splits of 5:1:1, 4\.5:1:1, and 4:1:1, respectively\. All observations from the same subject or patient remain in one split, and PAD\-UFES\-20 predictions are aggregated at the lesion level\. OASIS\-3\(LaMontagneet al\.[2019](https://arxiv.org/html/2607.27289#bib.bib45)\)is reserved for frozen ADNI\-to\-OASIS\-3 evaluation without adaptation\.

#### Evaluation protocol\.

Results are averaged over three prespecified seeds using identical grouped splits, and no seed is selected or replaced based on test performance\. All model selection and post\-hoc calibration are performed on the validation set and frozen before test evaluation\. For each method and seed, scalar temperature scaling is fitted by minimizing validation negative log\-likelihood\.

We report Macro\-F1 as the primary predictive metric, accompanied by Area Under the Receiver Operating Characteristic Curve \(AUROC\), Brier score\(Brier[1950](https://arxiv.org/html/2607.27289#bib.bib43)\), and Expected Calibration Error \(ECE\)\(Guoet al\.[2017](https://arxiv.org/html/2607.27289#bib.bib37)\)\. Balanced accuracy is additionally reported for cross\-cohort evaluation\. Modality\-risk metrics are computed on the held\-out ADNI test set using the frozen final unimodal predictors\. ERCE is the mean absolute gap between predicted risk and realized unimodal Brier loss over ten equal\-frequency bins; High\-Loss AUROC defines high loss using the 80th percentile of the pooled validation\-set unimodal Brier losses; and Correct\-Modality Risk Accuracy \(CMRA\) is the fraction of ADNI test subjects with exactly one correct unimodal prediction for which the correct modality receives strictly lower predicted risk than the incorrect modality\.

For robustness evaluation, all methods are trained only on clean data\. The same fixed severity grid of controlled T1w and DTI degradation is applied to the held\-out ADNI test inputs for every method\. Retention AUC integrates clean\-normalized Macro\-F1 over degradation severity\. Missing F1 is the unweighted mean of Macro\-F1 under the T1w\-missing and DTI\-missing conditions\. Conflict F1 is evaluated on subjects whose two unimodal argmax predictions disagree\. Harm Rate is the proportion of incorrect fused predictions among subjects for which at least one unimodal predictor is correct\. Clean Harm is evaluated on unmodified ADNI, whereas Stress Harm aggregates the degradation, missingness, and conflict evaluations\.

### In\-Domain Predictive and Probabilistic Performance

Table[1](https://arxiv.org/html/2607.27289#Sx3.T1)comparesTIER\-MoEwith unimodal baselines, conventional fusion \(DAFT\(Pölsterlet al\.[2021](https://arxiv.org/html/2607.27289#bib.bib44)\); GMU\(Arevaloet al\.[2017](https://arxiv.org/html/2607.27289#bib.bib29)\)\), adaptive fusion \(QMF\(Zhanget al\.[2023](https://arxiv.org/html/2607.27289#bib.bib33)\); PDF\(Caoet al\.[2024](https://arxiv.org/html/2607.27289#bib.bib15)\)\), and expert routing \(Flex\-MoE\(Yunet al\.[2024](https://arxiv.org/html/2607.27289#bib.bib19)\);I2I^\{2\}MoE\(Xinet al\.[2025](https://arxiv.org/html/2607.27289#bib.bib20)\)\) across the three in\-domain datasets\.TIER\-MoEachieves the best Macro\-F1 and Brier score on every dataset and ranks first on 10 of the 12 dataset–metric combinations\. On ADNI, it reaches a Macro\-F1 of 0\.648, a 3\.4\-percentage\-point improvement over the strongest published comparator\. The same advantage extends to distinct modality regimes: Macro\-F1 reaches 0\.884 for image–clinical fusion on PAD\-UFES\-20 and 0\.657 for fundus–video fusion on FPRM\. On PAD\-UFES\-20 and FPRM,TIER\-MoEalso obtains the strongest AUROC, Brier score, and ECE\.

To isolate whether the gains arise from cross\-fitted predictive evidence alone or from affinity\-based expert routing alone, we compare two matched controls\. OOF MetaGate receives the same cross\-fitted evidence as the conditional\-risk estimator but directly learns sample\-specific modality weights without the expert\-routing branch\.TIER\-MoEimproves its Macro\-F1 by 4\.1%, 5\.8%, and 5\.5% on ADNI, PAD\-UFES\-20, and FPRM, respectively\. Affinity\-MoE is an alignment\-only subspace MoE that routes modality–expert pairs using expert\-subspace affinityψi,m,e\\psi\_\{i,m,e\}without the conditional modality risk\. Our method further improves both Macro\-F1 and Brier score over it on all three tasks\. Together, these comparisons isolate the value of both components; modeling cross\-fitted evidence as task\-grounded modality risk rather than direct fusion weights, and combining this risk with expert compatibility rather than routing by affinity alone\.

### Calibration and Discrimination of Conditional Modality Risk

Having established the downstream prediction gains, we next test the central quantity behind them: whether the estimated modality risk actually predicts the loss incurred by relying on that modality\.

![Refer to caption](https://arxiv.org/html/2607.27289v1/Table2.png)Figure 3:Modality\-risk semantics on held\-out ADNI\. ERCE andρs\\rho\_\{s\}measure agreement with realized unimodal Brier loss; AUROC detects high\-loss cases; CMRA evaluates correct\-modality ranking\.Figure[3](https://arxiv.org/html/2607.27289#Sx3.F3)compares OOF conditional risk with technical\-quality, confidence\-based, adaptive\-fusion, and in\-sample\-risk alternatives\. OOF conditional risk achieves the best result on all four metrics: it reduces ERCE from 0\.081 for PDF to 0\.050, raises Spearman correlation from 0\.309 for PDF to 0\.421, and improves High\-Loss AUROC from 0\.701 for QMF to 0\.774\. These results show that the estimated risk is both calibrated to realized loss and discriminative of high\-loss modality predictions\.

CMRA tests whether the risk score ranks the correct unimodal predictor below the incorrect one when exactly one of them is correct\. OOF conditional risk assigns lower risk to the correct modality in 74\.1% of these cases, compared with 66\.1% for PDF\. Together, the in\-sample control and the proxy\-score baselines isolate the benefit of cross\-fitted loss supervision over in\-sample risk fitting and technical\-quality, confidence, or fusion\-derived reliability signals\.

### Robustness to Unreliable Evidence

The preceding analysis shows that conditional risk tracks held\-out unimodal loss\. We next evaluate whetherTIER\-MoEremains robust when multimodal evidence is degraded, unavailable, or conflicting\.

Table 2:Robustness on ADNI under progressive modality degradation, single\-modality missingness, and natural unimodal disagreement\. Retention AUC integrates clean\-normalized Macro\-F1 over degradation severity\.TIER\-MoEpreserves performance under modality degradation\.Table[2](https://arxiv.org/html/2607.27289#Sx3.T2)shows thatTIER\-MoEachieves retention AUCs of 0\.902 and 0\.921 under progressive T1w and DTI degradation, exceeding PDF, the strongest baseline by 5\.1 and 4\.6 percentage points, respectively\. The advantage holds when either modality is degraded, including the globally stronger T1w modality, indicating that the model does not depend on a fixed modality preference\.

TIER\-MoEremains robust to missingness and unimodal disagreement\.TIER\-MoEattains the highest Missing F1 of 0\.531 and Conflict F1 of 0\.574\. These metrics probe complementary failure modes: Missing F1 measures prediction using the remaining modality when one source is unavailable, whereas Conflict F1 measures fusion performance when the two unimodal predictors produce competing decisions\. The strong results in both cases indicate thatTIER\-MoEcan operate effectively with incomplete evidence while resolving cross\-modal disagreement without consistently favoring one modality\.

TIER\-MoEfurther reduces Stress Harm from 0\.158 for PDF to 0\.109, a 31\.0% relative reduction\. Together with the retention results, these findings show thatTIER\-MoEdegrades more gracefully and is less likely to override correct unimodal evidence under modality degradation, missingness, and disagreement\.

### Zero\-Shot ADNI\-to\-OASIS\-3 Transfer

We next examine whether the learned reliability and routing mechanisms transfer to an external cohort without adaptation\. Under the frozen ADNI\-to\-OASIS\-3 protocol, Table[3](https://arxiv.org/html/2607.27289#Sx3.T3)shows thatTIER\-MoEachieves the best balanced accuracy \(0\.568\), Macro\-F1 \(0\.535\), Brier score \(0\.249\), and ECE \(0\.078\)\. Its AUROC of 0\.776 is within 0\.004 of the best result\. The strongest external results therefore occur in class\-balanced prediction and probability quality, while ranking performance remains competitive\.

The transfer extends beyond the final prediction to the learned risk semantics\. Without OASIS\-3 training or recalibration, OOF conditional risk reduces ERCE from 0\.101 for PDF to 0\.074, raises Spearman correlation from 0\.261 for PDF to 0\.347, improves High\-Loss AUROC from 0\.671 for QMF to 0\.724, and increases CMRA from 0\.626 for PDF to 0\.684\. Thus, the estimated risks retain their calibration to realized modality loss, high\-loss discrimination, and correct\-modality ordering under cohort shift\. The zero\-shot transfer therefore extends beyond downstream classification to the task\-grounded reliability semantics central toTIER\-MoE\.

### Ablation Studies

Finally, we use controlled ablations to examine three design questions: whether the reliability and interaction branches provide complementary capabilities; whether conditional modality risk improves affinity\-based expert routing; and whether OOF supervision is necessary for task\-grounded risk estimation\. The first three variants evaluate frozen branches or pathways from the same trained model, whereas the remaining variants modify the routing criterion or risk supervision\.

The prediction branches exhibit complementary strengths\.Table[4](https://arxiv.org/html/2607.27289#Sx3.T4)shows that the interaction branch achieves higher Macro\-F1 than the reliability anchor \(0\.621 vs\. 0\.603\) and a lower Brier score \(0\.204 vs\. 0\.227\), whereas the reliability anchor yields lower Clean Harm \(0\.126 vs\. 0\.153\)\. Within the interaction branch, removing the sparse expert residuals reduces Macro\-F1 from 0\.621 to 0\.615 and worsens Brier score from 0\.204 to 0\.211, showing that the routed experts add predictive value beyond the shared path\. The shared path alone produces slightly lower Clean Harm than the full interaction branch \(0\.148 vs\. 0\.153\), indicating a trade\-off between specialized interaction and decision stability\. Combining the reliability and interaction predictions with the validation\-selectedγ\\gammaachieves the best Macro\-F1 \(0\.648\), Brier score \(0\.189\), and Clean Harm \(0\.091\)\.

Risk\-aware routing and OOF supervision both contribute to the full model\.Removing conditional risk from the routing score by settingη=0\\eta=0reduces Macro\-F1 from 0\.648 to 0\.633, worsens Brier score from 0\.189 to 0\.199, and increases Clean Harm from 0\.091 to 0\.114\. Separately, replacing the OOF\-derived risk targets with in\-sample supervision yields a Macro\-F1 of 0\.629, a Brier score of 0\.205, and a Clean Harm of 0\.121\. These comparisons isolate the contributions of risk\-conditioned expert routing and label\-honest OOF supervision to predictive performance, probability quality, and harmful\-fusion control\.

Table 3:Frozen ADNI\-to\-OASIS\-3 cross\-cohort evaluation\. All model hyperparameters are selected using ADNI only and applied to OASIS\-3 without adaptation\.Table 4:Component and routing ablations on ADNI\. The first three rows evaluate frozen branches or pathways from the same trained model;η=0\\eta=0removes conditional risk from the routing score, and the in\-sample variant replaces OOF\-derived risk targets\.

## Conclusion

We introduceTIER\-MoE, which learns subject\-specific modality risk from out\-of\-fold losses, combines it with expert\-subspace affinity for sparse routing, and preserves cross\-modal interaction through a shared path\. Across three in\-domain tasks,TIER\-MoEachieves the best Macro\-F1 and Brier score; its risks track realized unimodal loss, and it remains robust to degradation, missingness, and disagreement, with strong zero\-shot transfer to OASIS\-3\. These results show that modality reliability, cross\-modal complementarity, and expert compatibility are distinct yet coordinated factors in multimodal fusion\.

## References

- Multimodal biomedical AI\.Nature Medicine28\(9\),pp\. 1773–1784\.External Links:[Document](https://dx.doi.org/10.1038/s41591-022-01981-2)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1)\.
- J\. Arevalo, T\. Solorio, M\. Montes\-y\-Gómez, and F\. A\. González \(2017\)Gated multimodal units for information fusion\.arXiv preprint arXiv:1702\.01992\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1),[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.
- G\. W\. Brier \(1950\)Verification of forecasts expressed in terms of probability\.Monthly Weather Review78\(1\),pp\. 1–3\.Cited by:[Evaluation protocol\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px2.p2.1)\.
- E\. E\. Bron, M\. Smits, J\. M\. Papma, R\. M\. E\. Steketee, R\. Meijboom, M\. de Groot, J\. C\. van Swieten, W\. J\. Niessen, and S\. Klein \(2017\)Multiparametric computer\-aided differential diagnosis of alzheimer’s disease and frontotemporal dementia using structural and advanced MRI\.European Radiology27,pp\. 3372–3382\.External Links:[Document](https://dx.doi.org/10.1007/s00330-016-4691-x)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1)\.
- B\. Cao, Y\. Xia, Y\. Ding, C\. Zhang, and Q\. Hu \(2024\)Predictive dynamic fusion\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 5608–5628\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1),[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.
- A\. Cheng, S\. Duan, S\. Li, C\. Yin, M\. Cheng, H\. Ping, T\. Chattopadhyay, S\. I\. Thomopoulos, S\. Nazarian, P\. Thompson, and P\. Bogdan \(2026\)ERMoE: eigen\-reparameterized mixture\-of\-experts for stable routing and interpretable specialization\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 12997–13006\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Cheng_ERMoE_Eigen-Reparameterized_Mixture-of-Experts_for_Stable_Routing_and_Interpretable_Specialization_CVPR_2026_paper.html)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1),[Risk\-Guided Subspace Expert Routing](https://arxiv.org/html/2607.27289#Sx2.SSx3.p2.4),[Interaction objective\.](https://arxiv.org/html/2607.27289#Sx2.SSx4.SSS0.Px1.p1.3)\.
- L\. M\. T\. Doan, K\. Shahhosseini, S\. Verma, A\. Marefat, G\. Locicero, S\. Verma, C\. Angione, and A\. Occhipinti \(2026\)Bridging modalities with AI: a review of AI advances in multimodal biomedical imaging\.Communications Engineering5,pp\. 30\.External Links:[Document](https://dx.doi.org/10.1038/s44172-026-00602-x)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1)\.
- C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger \(2017\)On calibration of modern neural networks\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 1321–1330\.Cited by:[Evaluation protocol\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px2.p2.1)\.
- Z\. Han, F\. Yang, J\. Huang, C\. Zhang, and J\. Yao \(2022\)Multimodal dynamics: dynamical fusion for trustworthy multimodal classification\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 20675–20685\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.02005)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- Z\. Han, C\. Zhang, H\. Fu, and J\. T\. Zhou \(2023\)Trusted multi\-view classification with dynamic evidential fusion\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(2\),pp\. 2551–2566\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2022.3171983)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- Y\. Huang, J\. Lin, C\. Zhou, H\. Yang, and L\. Huang \(2022\)Modality competition: what makes joint training of multi\-modal network fail in deep learning? \(provably\)\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 9226–9259\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- P\. J\. LaMontagne, T\. L\. S\. Benzinger, J\. C\. Morris, S\. Keefe, R\. Hornbeck, C\. Xiong, E\. Grant, J\. Hassenstab, K\. Moulder, A\. G\. Vlassenko, M\. E\. Raichle, C\. Cruchaga, and D\. Marcus \(2019\)OASIS\-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease\.medRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2019.12.13.19014902)Cited by:[Datasets and splits\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px1.p1.1)\.
- S\. Lorio, F\. Kherif, A\. Ruef, L\. Melie\-Garcia, R\. S\. J\. Frackowiak, J\. Ashburner, G\. Helms, A\. Lutti, and B\. Draganski \(2016\)Neurobiological origin of spurious brain morphological changes: a quantitative MRI study\.Human Brain Mapping37\(5\),pp\. 1801–1815\.External Links:[Document](https://dx.doi.org/10.1002/hbm.23137)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1)\.
- A\. Nagrani, S\. Yang, A\. Arnab, A\. Jansen, C\. Schmid, and C\. Sun \(2021\)Attention bottlenecks for multimodal fusion\.InAdvances in Neural Information Processing Systems,Vol\.34,pp\. 14200–14213\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- N\. Neverova, C\. Wolf, G\. W\. Taylor, and F\. Nebout \(2016\)ModDrop: adaptive multi\-modal gesture recognition\.IEEE Transactions on Pattern Analysis and Machine Intelligence38\(8\),pp\. 1692–1706\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2015.2461544)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- A\. G\. C\. Pacheco, G\. R\. Lima, A\. S\. Salomão, B\. A\. Krohling, I\. P\. Biral, G\. G\. de Angelo, F\. C\. R\. Alves, J\. G\. M\. Esgario, A\. C\. Simora, P\. B\. C\. Castro, F\. B\. Rodrigues, P\. H\. L\. Frasson, R\. A\. Krohling, H\. Knidel, M\. C\. S\. Santos, R\. B\. do Espirito Santo, T\. L\. S\. G\. Macedo, T\. R\. P\. Canuto, and L\. F\. S\. de Barros \(2020\)PAD\-UFES\-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones\.Data in Brief32,pp\. 106221\.External Links:[Document](https://dx.doi.org/10.1016/j.dib.2020.106221)Cited by:[Datasets and splits\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px1.p1.1)\.
- R\. C\. Petersen, P\. S\. Aisen, L\. A\. Beckett, M\. C\. Donohue, A\. C\. Gamst, D\. J\. Harvey, C\. R\. Jack, W\. J\. Jagust, L\. M\. Shaw, A\. W\. Toga, J\. Q\. Trojanowski, and M\. W\. Weiner \(2010\)Alzheimer’s disease neuroimaging initiative \(ADNI\): clinical characterization\.Neurology74\(3\),pp\. 201–209\.External Links:[Document](https://dx.doi.org/10.1212/WNL.0b013e3181cb3e25)Cited by:[Datasets and splits\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px1.p1.1)\.
- S\. Pölsterl, T\. N\. Wolf, and C\. Wachinger \(2021\)Combining 3d image and tabular data via the dynamic affine feature map transform\.InMedical Image Computing and Computer Assisted Intervention – MICCAI 2021,Lecture Notes in Computer Science, Vol\.12905,pp\. 688–698\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-87240-3%5F66)Cited by:[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.
- K\. Poulakis, J\. B\. Pereira, J\. Muehlboeck, L\. Wahlund, Ö\. Smedby, G\. Volpe, C\. L\. Masters, D\. Ames, Y\. Niimi, T\. Iwatsubo, D\. Ferreira, E\. Westman,et al\.\(2022\)Multi\-cohort and longitudinal bayesian clustering study of stage and subtype in alzheimer’s disease\.Nature Communications13,pp\. 4566\.External Links:[Document](https://dx.doi.org/10.1038/s41467-022-32202-6)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1)\.
- C\. Riquelme, J\. Puigcerver, B\. Mustafa, M\. Neumann, R\. Jenatton, A\. S\. Pinto, D\. Keysers, and N\. Houlsby \(2021\)Scaling vision with sparse mixture of experts\.InAdvances in Neural Information Processing Systems,Vol\.34,pp\. 8583–8595\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1)\.
- N\. Shazeer, A\. Mirhoseini, K\. Maziarz, A\. Davis, Q\. V\. Le, G\. Hinton, and J\. Dean \(2017\)Outrageously large neural networks: the sparsely\-gated mixture\-of\-experts layer\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1)\.
- S\. Sidheekh, P\. Tenali, S\. Mathur, E\. Blasch, K\. Kersting, and S\. Natarajan \(2025\)Credibility\-aware multimodal fusion using probabilistic circuits\.InProceedings of the 28th International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.258,pp\. 2305–2313\.External Links:[Link](https://proceedings.mlr.press/v258/sidheekh25a.html)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- S\. R\. Stahlschmidt, B\. Ulfenborg, and J\. Synnergren \(2022\)Multimodal machine learning for biomedical applications: a review\.Briefings in Bioinformatics23\(6\),pp\. bbac408\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1)\.
- W\. Tae, B\. Ham, S\. Pyun, S\. Kang, and B\. Kim \(2018\)Current clinical applications of diffusion\-tensor imaging in neurological disorders\.Journal of Clinical Neurology14\(2\),pp\. 129–140\.External Links:[Document](https://dx.doi.org/10.3988/jcn.2018.14.2.129)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1)\.
- W\. Wang, D\. Tran, and M\. Feiszli \(2020\)What makes training multi\-modal classification networks hard?\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 12695–12705\.External Links:[Document](https://dx.doi.org/10.1109/CVPR42600.2020.01271)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1)\.
- C\. Wu, Z\. Shuai, Z\. Tang, L\. Wang, and L\. Shen \(2025\)Dynamic modeling of patients, modalities and tasks via multi\-modal multi\-task mixture of experts\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1)\.
- J\. Xin, S\. Yun, J\. Peng, I\. Choi, J\. L\. Ballard, T\. Chen, and Q\. Long \(2025\)I2I^\{2\}MoE: interpretable multimodal interaction\-aware mixture\-of\-experts\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 68870–68888\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1),[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.
- S\. Yun, I\. Choi, J\. Peng, Y\. Wu, J\. Bao, Q\. Zhang, J\. Xin, Q\. Long, and T\. Chen \(2024\)Flex\-MoE: modeling arbitrary modality combination via the flexible mixture\-of\-experts\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 98782–98805\.External Links:[Document](https://dx.doi.org/10.52202/079017-3135)Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p3.1),[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.
- G\. Zhang, Y\. Qu, Y\. Zhang, J\. Tang, C\. Wang, H\. Yin, X\. Yao, G\. Liang, T\. Shen, Q\. Ren, H\. Jia, and X\. Sun \(2024\)Multimodal eye imaging, retina characteristics, and psychological assessment dataset\.Scientific Data11\(1\),pp\. 836\.External Links:[Document](https://dx.doi.org/10.1038/s41597-024-03690-6)Cited by:[Datasets and splits\.](https://arxiv.org/html/2607.27289#Sx3.SSx1.SSS0.Px1.p1.1)\.
- Q\. Zhang, H\. Wu, C\. Zhang, Q\. Hu, H\. Fu, J\. T\. Zhou, and X\. Peng \(2023\)Provable dynamic fusion for low\-quality multimodal data\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 41753–41769\.Cited by:[Introduction](https://arxiv.org/html/2607.27289#Sx1.p1.1),[Introduction](https://arxiv.org/html/2607.27289#Sx1.p2.1),[In\-Domain Predictive and Probabilistic Performance](https://arxiv.org/html/2607.27289#Sx3.SSx2.p1.1)\.