Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients

# Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients
Source: [https://arxiv.org/html/2608.21941](https://arxiv.org/html/2608.21941)
Yueyang SunThanks:Equal ContributionAffiliation:International School, Beijing University of Posts and Telecommunications,Weichen LiuAffiliation:School of Computer Science and Engineering, Southeast University,Xianbing ZhaoAffiliation:School of Artificial Intelligence and Computer Science, Jiangnan Universityyyx@smbu\.edu\.cn, 2474931434@bupt\.edu\.cn, 213233033@seu\.edu\.cn,zhaoxianbing\_hitsz@163\.com, liusc@smbu\.edu\.cnSicen LiuThanks:Corresponding AuthorAffiliation:Faculty of Engineering, Shenzhen MSU\-BIT University,

###### Abstract

Accurate assessment of patients in intensive care units \(ICUs\) is essential for timely clinical intervention and improved patient outcomes\. Multimodal electronic health records \(EHRs\), including structured physiological time series and longitudinal clinical notes, provide complementary information for critical care prediction\. However, in real\-world clinical settings, individual modalities may be partially observed or entirely unavailable, resulting in substantial performance degradation for existing multimodal models\. To address this challenge, we propose a multimodal prompt\-learning framework for robust clinical prediction under diverse missing\-modality scenarios\. The proposed framework introduces four complementary types of prompts: generative prompts, missing\-signal prompts, missing\-type prompts, and temporal prompts\. Generative prompts construct surrogate latent representations for unavailable modalities, while missing\-signal prompts distinguish observed representations from generated ones\. Missing\-type prompts condition the model on different modality\-availability configurations, whereas temporal prompts perform condition\-specific aggregation over temporally encoded clinical sequences\. Together, these prompts enable the model to capture missingness\-aware intramodal dependencies and cross\-modal interactions within a unified architecture\. Extensive experiments demonstrate that our method outperforms existing approaches across evaluation metrics on two missingness settings\. Ablation and robustness analyses further verify the complementary contributions of the four prompt types and the effectiveness of the proposed framework for clinical prediction from incomplete multimodal EHR data\.

## 1Introduction

Intensive care units \(ICUs\) provide continuous care for patients with life\-threatening conditions, such as severe trauma\[[33](https://arxiv.org/html/2608.21941#bib.bib2)\], sepsis\[[4](https://arxiv.org/html/2608.21941#bib.bib3)\], and organ failure\[[2](https://arxiv.org/html/2608.21941#bib.bib4)\]\. The early hours following ICU admission are particularly critical, as patients may experience rapid physiological deterioration and are highly vulnerable to delayed or inappropriate clinical decisions\[[8](https://arxiv.org/html/2608.21941#bib.bib5),[25](https://arxiv.org/html/2608.21941#bib.bib6)\]\. Accurate prediction of in\-hospital mortality based on patients’ clinical information collected during the first 48 hours of ICU admission is therefore essential for timely risk assessment, resource allocation, and clinical intervention\. The widespread adoption of electronic health records \(EHRs\) has substantially transformed the documentation and analysis of patients’ health conditions in ICUs\[[1](https://arxiv.org/html/2608.21941#bib.bib7),[5](https://arxiv.org/html/2608.21941#bib.bib8),[29](https://arxiv.org/html/2608.21941#bib.bib9)\]\. EHRs contain heterogeneous and complementary sources of clinical information, including multivariate irregularly sampled time series \(MISTS\), such as laboratory measurements and vital signs, as well as longitudinal clinical notes documenting patients’ symptoms, diagnoses, treatments, and disease progression\. The increasing availability of these data has facilitated the development of deep learning models for various clinical applications\[[41](https://arxiv.org/html/2608.21941#bib.bib10),[3](https://arxiv.org/html/2608.21941#bib.bib11),[30](https://arxiv.org/html/2608.21941#bib.bib12)\], including mortality prediction\[[42](https://arxiv.org/html/2608.21941#bib.bib13)\], sepsis onset prediction\[[34](https://arxiv.org/html/2608.21941#bib.bib14)\],phenotype classification\[[14](https://arxiv.org/html/2608.21941#bib.bib15)\], and medication recommendation\[[47](https://arxiv.org/html/2608.21941#bib.bib16),[24](https://arxiv.org/html/2608.21941#bib.bib17)\]\.

![Refer to caption](https://arxiv.org/html/2608.21941v1/missing_case.png)Figure 1:Examples of different types of missing information in EHR data include missing clinical notes, missing MISTS, and missing data within the modality\. The heart rate and temperature are monitored regularly with different frequencies, and glucose is a laboratory test ordered at irregular time intervals\. The clinical notes are free text\.Existing studies have developed multimodal architectures to integrate irregular physiological measurements and clinical notes\[[46](https://arxiv.org/html/2608.21941#bib.bib20),[23](https://arxiv.org/html/2608.21941#bib.bib22)\]\. However, both modalities may not always be available during inference, as real\-world EHRs may contain incomplete data or have one modality entirely missing due to equipment failures, delayed documentation, data corruption, privacy restrictions, or variations in clinical workflows\. As illustrated in Figure[1](https://arxiv.org/html/2608.21941#S1.F1), different missing\-modality configurations introduce distinct information gaps and may substantially degrade mortality prediction\. Moreover, observations within each modality are temporally irregular: physiological variables are recorded at uneven intervals, while clinical notes are documented at different stages of an ICU stay\. Therefore, a robust model should distinguish observed representations from generated ones, adapt its fusion strategy to different modality\-availability conditions, and preserve the irregular temporal information encoded in the available clinical streams\.

Prompt learning provides a flexible mechanism for adapting model representations to different input conditions through learnable parameters\[[10](https://arxiv.org/html/2608.21941#bib.bib18),[18](https://arxiv.org/html/2608.21941#bib.bib19)\]\. Motivated by this property, we propose a multimodal prompt\-learning framework for in\-hospital mortality prediction using EHR data collected during the first 48 hours of ICU admission\. The framework incorporates four complementary prompts\. Generative prompts provide surrogate representations for unavailable modalities, missing\-signal prompts distinguish observed features from surrogate representations, missing\-type prompts characterize modality\-availability configurations, and temporal prompts perform condition\-specific aggregation over the irregular temporal information\. These prompts are jointly integrated into a multimodal Transformer to learn missingness\-aware temporal, intramodal, and cross\-modal representations from incomplete EHR data\.

We conduct extensive experiments on two large publicly available benchmark datasets, MIMIC\-III\[[16](https://arxiv.org/html/2608.21941#bib.bib24)\]and MIMIC\-IV\[[15](https://arxiv.org/html/2608.21941#bib.bib1)\], validating the effectiveness and superiority of our proposed method\.

The main contributions of this work are three\-fold:

- •We present a novel framework via prompt learning for monitoring critical care patients that is capable of handling missing modalities efficiently with irregular EHR\.
- •We propose four types of prompts to tackle the problem of missing modalities in irregular EHR data\. These prompts can generate missing information, learn irregular temporal patterns, and effectively capture both intra\- and inter\-modality information\.
- •Extensive experiments on large public available benchmark dataset MIMIC\-III and MIMIC\-IV dataset demonstrate that our proposed approach outperforms the compared methods\.

## 2Related Work

Multimodal Modeling of Irregular EHRs\.Electronic health records commonly contain structured multivariate irregular time series \(MISTS\) and temporally irregular clinical notes\. Earlier studies address irregularity through time\-aware decay, learnable interpolation, set\-based representations, or attention over continuous\-time embeddings\[[7](https://arxiv.org/html/2608.21941#bib.bib26),[31](https://arxiv.org/html/2608.21941#bib.bib25),[13](https://arxiv.org/html/2608.21941#bib.bib44),[32](https://arxiv.org/html/2608.21941#bib.bib23)\]\. Recent research has increasingly focused on jointly modeling structured and unstructured EHR modalities\. Zhang et al\. propose a unified irregular\-EHR framework that models MISTS and clinical\-note sequences separately and integrates them through temporally interleaved attention\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]\. Liu and Chen further incorporate domain and memory knowledge into irregular multimodal EHR representations\[[23](https://arxiv.org/html/2608.21941#bib.bib22)\], while CTPD discovers and aligns cross\-modal temporal patterns shared across patients\[[36](https://arxiv.org/html/2608.21941#bib.bib35)\]\. These approaches improve temporal representation and multimodal fusion, but primarily focus on extracting information from the available input streams\.

Learning with Missing Clinical Modalities\.Missing modalities are common in clinical data because measurements, notes, images, and reports are collected under heterogeneous clinical workflows\. Earlier work explored missing\-modality learning through cross\-modal translation, modality imagination, and missing\-modality\-aware prompting\[[27](https://arxiv.org/html/2608.21941#bib.bib32),[48](https://arxiv.org/html/2608.21941#bib.bib33),[19](https://arxiv.org/html/2608.21941#bib.bib34)\]\. Recent clinical studies consider more realistic settings: MUSE jointly addresses missing modalities and labels\[[39](https://arxiv.org/html/2608.21941#bib.bib47)\]\. Wang et al\. address missing chest radiographs, radiology reports, or structured clinical variables\[[37](https://arxiv.org/html/2608.21941#bib.bib48)\], while Liang et al\. model missing patterns related to patient conditions or clinical decisions\[[21](https://arxiv.org/html/2608.21941#bib.bib41)\]\. More recently, Zhao et al\. combine modality reconstruction with prompt\-guided adaptation to handle missing MRI sequences during both training and inference\[[49](https://arxiv.org/html/2608.21941#bib.bib49)\]\. Despite these advances, existing methods primarily emphasize latent compensation, representation consistency, or robust aggregation\. Our work instead couples prompt\-conditioned cross\-modal generation with explicit representations of latent\-stream provenances, whole\-modality availability, and irregular temporal progression for paired MISTS and longitudinal clinical notes\.

Prompt Learning for Missing Modalities\.Prompt\-based adaptation uses task\-specific natural\-language templates in NLP and learnable multimodal prompt tokens in vision\-language models\[[10](https://arxiv.org/html/2608.21941#bib.bib18),[18](https://arxiv.org/html/2608.21941#bib.bib19)\]\. Lee et al\. introduces multimodal prompts for visual recognition with missing inputs\[[19](https://arxiv.org/html/2608.21941#bib.bib34)\]\. Most closely related to our work, Guo et al\. propose generative, missing\-signal, and missing\-type prompts for multimodal sentiment analysis and emotion recognition under missing modalities\[[11](https://arxiv.org/html/2608.21941#bib.bib21)\]\. However, these methods are designed for general multimodal settings rather than EHR data, and none of them directly addresses missing modalities in irregular EHR with heterogeneous temporal patterns\. We adapt this prompt\-based missing\-modality paradigm to irregular clinical data by introducing temporal prompts and integrating prompt\-conditioned reconstruction with time\-aware MISTS and clinical\-note encoders for ICU outcome prediction\.

![Refer to caption](https://arxiv.org/html/2608.21941v1/framework.png)Figure 2:Overview of the proposed prompt\-based framework for robust outcome prediction from incomplete irregular multimodal EHRs, based on a Notes\-only example\. Given a notes\-only input condition, irregular notes encoders extract the available representation, and the Missing Modality Generation Module reconstructs the absent MISTS stream\. Missing\-signal prompts augment each token with a provenance embedding indicating whether its modality stream is observed or prompt\-generated, while missing\-type prompts inject the sample\-level modality\-availability condition into both post\-fusion streams\. The resulting representations are fused and summarized by temporal prompts for patient outcome prediction\.
## 3Method

As illustrated in Figure[2](https://arxiv.org/html/2608.21941#S2.F2), the proposed framework takes multivariate irregularly sampled time series and longitudinal clinical notes as input\. Four types of learnable prompts are incorporated into a multimodal Transformer to model irregular temporal information and diverse missing\-modality patterns\. The resulting multimodal representation is subsequently fed into a prediction head to estimate the probability of in\-hospital mortality\.

### 3\.1Problem Formulation

We formulate critical care monitoring as a patient\-level prediction problem from irregular multimodal EHRs\. Each sample from an ICU stay is observed within an early clinical window, that is, the first 48 hours after admission\. The sample index is omitted for clarity\. The EHR data comprise two heterogeneous modalities:v∈\{M,N\}v\\in\\\{M,N\\\}, whereMMdenotes MISTS andNNdenotes the clinical notes\. Letm∈\{c,M​m,N​m\}m\\in\\\{c,Mm,Nm\\\}denote the sample\-level modality condition, wherem=cm=cindicates that both modalities are observed,m=M​mm=Mmindicates MISTS is missing, andm=N​mm=Nmindicates notes is missing\. The input can be formulated as

X=\{\{\(tiM,xiM\)\}i=1TM,\{\(tjN,xjN\)\}j=1TN,m\}X=\\\{\\\{\(t\_\{i\}^\{M\},x^\{M\}\_\{i\}\)\\\}\_\{i=1\}^\{T\_\{M\}\},\\\{\(t\_\{j\}^\{N\},x^\{N\}\_\{j\}\)\\\}\_\{j=1\}^\{T\_\{N\}\},m\\\}\(1\)with observation timetkvt\_\{k\}^\{v\}, multivariate clinical valuexkvx^\{v\}\_\{k\}, and number of observed eventsTvT\_\{v\}\. For a missing modality, the corresponding input sequence is unavailable\. The objective is to learny^=fθ​\(X\)∈\[0,1\]\\hat\{y\}=f\_\{\\theta\}\(X\)\\in\[0,1\], the predicted risk of 48\-hour in\-hospital mortality\.

### 3\.2Missing Modality Generation Module

Since MISTS and clinical notes differ in their structures and temporal characteristics, we employ separate modality\-specific encoders to obtain their latent representations when they are available: the UTDE encoder\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]for MISTS and the Clinical\-Longformer\-based mTANDtxtencoder\[[20](https://arxiv.org/html/2608.21941#bib.bib46),[46](https://arxiv.org/html/2608.21941#bib.bib20)\]for clinical notes\.

HM=E​n​cM​\(XM,TM\)H^\{M\}=\{Enc\}\_\{M\}\(X^\{M\},T^\{M\}\)\(2\)HN=E​n​cN​\(XN,TN\)H^\{N\}=\{Enc\}\_\{N\}\(X^\{N\},T^\{N\}\)\(3\)
whereXMX^\{M\},TMT^\{M\}denote the numerical observations, timestamps of MISTS, respectively, whileXNX^\{N\}andTNT^\{N\}denote the note embeddings and their timestamps\.

When one modality is missing, we employ the Missing Modality Generation Module to construct a surrogate representation for the unavailable modality\. As depicted in Figure[2](https://arxiv.org/html/2608.21941#S2.F2), when clinical notes are observed and MISTS are missing \(m=M​mm=Mm\),HNH^\{N\}is obtained directly from the irregular notes encoder, whileHMH^\{M\}is generated fromHNH^\{N\}using the generative promptPGMP\_\{G\}^\{M\}through cross\-attention\[[35](https://arxiv.org/html/2608.21941#bib.bib43)\]:

HM=CrossAttn⁡\(PGM,HN,HN\)\.H^\{M\}=\\operatorname\{CrossAttn\}\\left\(P\_\{G\}^\{M\},H^\{N\},H^\{N\}\\right\)\.\(4\)
Here,PGM∈ℝL×dP\_\{G\}^\{M\}\\in\\mathbb\{R\}^\{L\\times d\}is a learnable generative prompt, whereLLdenotes the number of positions on the unified temporal grid anddddenotes the latent dimension shared by the two modalities\.PGMP\_\{G\}^\{M\}serves as the query, while the observed clinical\-note representationHNH^\{N\}serves as both the keys and values\. The resultingHMH^\{M\}is a time\-aligned surrogate representation for the missing MISTS modality\.

### 3\.3Missing Signal Prompt Module

Although the generated representationHMH^\{M\}restores the missing MISTS stream, it should not be treated in the same way as the directly observed clinical\-note representationHNH^\{N\}\. We therefore introduce the Missing Signal Prompt Module to indicate whether each modality representation is observed or generated\.

The module annotates each stream with its provenance using four learnable prompts for both modalities:PSM​m,PSM​o,PSN​m,PSN​o∈ℝL×dP^\{Mm\}\_\{S\},P^\{Mo\}\_\{S\},P^\{Nm\}\_\{S\},P^\{No\}\_\{S\}\\in\\mathbb\{R\}^\{L\\times d\}, indicating that the modality is missing \(M​m,N​mMm,Nm\) or observed \(M​o,N​oMo,No\)\. Therefore, the corresponding prompts are added as follows:

H~M\\displaystyle\\widetilde\{H\}^\{M\}=HM\+PSM​m,\\displaystyle=H^\{M\}\+P\_\{S\}^\{Mm\},\(5\)H~N\\displaystyle\\widetilde\{H\}^\{N\}=HN\+PSN​o\.\\displaystyle=H^\{N\}\+P\_\{S\}^\{No\}\.\(6\)
Here, by addingPSM​mP\_\{S\}^\{Mm\}andPSN​oP\_\{S\}^\{No\}to the generated MISTS representation and observed clinical\-note representation, respectively, each time\-aligned token now carries a position\-specific marker of its source, so that the downstream fusion is able to weigh the observed and synthesized features asymmetrically rather than blending them uniformly\.

### 3\.4Missing Type Prompt Module

Although the pair of provenance prompts implicitly identifies the modality condition, this information remains distributed across the two streams\. We therefore introduce a shared missing\-type prompt that provides both post\-fusion branches with an explicit global availability condition\.

Specifically, the Missing Type Prompt Module provides an explicit global condition through three learnable missing\-type prompts:PTc,PTM​m,PTN​m∈ℝLp×dP^\{c\}\_\{T\},P^\{Mm\}\_\{T\},\\,P^\{Nm\}\_\{T\}\\in\\mathbb\{R\}^\{L\_\{p\}\\times d\}, whereLpL\_\{p\}is the number of type tokens\. The same missing\-type prompt is used for both modality streams to provide a global condition\.

The two modality representations first exchange information through bidirectional cross\-attention:

CM\\displaystyle C^\{M\}=CrossAttn⁡\(H~M,H~N,H~N\),\\displaystyle=\\operatorname\{CrossAttn\}\\left\(\\widetilde\{H\}^\{M\},\\widetilde\{H\}^\{N\},\\widetilde\{H\}^\{N\}\\right\),\(7\)CN\\displaystyle C^\{N\}=CrossAttn⁡\(H~N,H~M,H~M\)\.\\displaystyle=\\operatorname\{CrossAttn\}\\left\(\\widetilde\{H\}^\{N\},\\widetilde\{H\}^\{M\},\\widetilde\{H\}^\{M\}\\right\)\.\(8\)
When modalityMMis missing,PTM​mP\_\{T\}^\{Mm\}is selected and appended to both cross\-attended representations:

ZM\\displaystyle Z^\{M\}=SelfAttnM⁡\(\[CM;PTM​m\]\),\\displaystyle=\\operatorname\{SelfAttn\}\_\{M\}\\left\(\[C^\{M\};P\_\{T\}^\{Mm\}\]\\right\),\(9\)ZN\\displaystyle Z^\{N\}=SelfAttnN⁡\(\[CN;PTM​m\]\)\.\\displaystyle=\\operatorname\{SelfAttn\}\_\{N\}\\left\(\[C^\{N\};P\_\{T\}^\{Mm\}\]\\right\)\.\(10\)
Here,\[⋅;⋅\]\[\\cdot\\,;\\cdot\]denotes sequence\-wise concatenation andZM,ZN∈ℝ\(L\+Lp\)×dZ^\{M\},Z^\{N\}\\in\\mathbb\{R\}^\{\(L\+L\_\{p\}\)\\times d\}\. The two self\-attention stacks share the same architecture but have separate parameters\. By usingPTM​mP\_\{T\}^\{Mm\}in both streams, the fusion module is explicitly informed that the MISTS representation is generated while the clinical\-note representation is observed\.

### 3\.5Temporal Prompt Module

The final prediction still requires a patient\-level summary over the irregular clinical timeline\. A fixed pooling operation cannot adaptively identify the temporal positions that are most relevant under different modality\-availability conditions\. We therefore introduce the Temporal Prompt Module for condition\-specific temporal aggregation\.

BecauseZMZ^\{M\}andZNZ^\{N\}contain both temporal tokens and the appended missing\-type prompt tokens, we retain their firstLLtemporal tokens and concatenate them along the feature dimension:

F=Concatfeat\(Z1:LM,Z1:LN\)∈ℝL×2​d\.F=\\operatorname\{Concat\}\_\{\\mathrm\{feat\}\}\\left\(Z^\{M\}\_\{1:L\},Z^\{N\}\_\{1:L\}\\right\)\\in\\mathbb\{R\}^\{L\\times 2d\}\.\(11\)
Here,ZM1:LZ^\{M\}\_\{1:L\}andZN1:LZ^\{N\}\_\{1:L\}denote the firstLLtime\-aligned modality tokens inZMZ^\{M\}andZNZ^\{N\}\. The remainingLpL\_\{p\}tokens correspond to the appended missing\-type prompts and are excluded from temporal aggregation\.Concatfeat\\operatorname\{Concat\}\_\{\\mathrm\{feat\}\}denotes concatenation along the feature dimension\.

To adapt temporal aggregation to different modality\-availability conditions, we maintain three learnable temporal prompts:Pτc,PτM​m,PτN​m∈ℝ1×2​dP^\{c\}\_\{\\tau\},\\,P^\{Mm\}\_\{\\tau\},\\,P^\{Nm\}\_\{\\tau\}\\in\\mathbb\{R\}^\{1\\times 2d\}, corresponding to three modality missing condition\. When modalityMMis missing,PτM​mP\_\{\\tau\}^\{Mm\}is selected:

sM​m=CrossAttn⁡\(PτM​m,F,F\)\.s\_\{Mm\}=\\operatorname\{CrossAttn\}\\left\(P\_\{\\tau\}^\{Mm\},F,F\\right\)\.\(12\)Here,PτM​mP\_\{\\tau\}^\{Mm\}serves as the query, whileFFprovides the keys and values\. The resulting representationsM​m∈ℝ1×2​ds\_\{Mm\}\\in\\mathbb\{R\}^\{1\\times 2d\}summarizes the temporal information most relevant to the Notes\-only condition\.

Finally, the summarized representation is processed by a residual two\-layer MLP and a linear classifier:

rM​m\\displaystyle r\_\{Mm\}=sM​m\+MLP⁡\(sM​m\),\\displaystyle=s\_\{Mm\}\+\\operatorname\{MLP\}\(s\_\{Mm\}\),\(13\)y^\\displaystyle\\widehat\{y\}=\[Softmax⁡\(Wo​rM​m\+bo\)\]mortality\\displaystyle=\\left\[\\operatorname\{Softmax\}\\left\(W\_\{o\}r\_\{Mm\}\+b\_\{o\}\\right\)\\right\]\_\{\\mathrm\{mortality\}\}\(14\)Here,rM​m∈ℝ2​d\{r\}\_\{Mm\}\\in\\mathbb\{R\}^\{2d\}denotes the resulting patient\-level representation after residual connection\.Wo∈ℝ2×2​dW\_\{o\}\\in\\mathbb\{R\}^\{2\\times 2d\}andbo∈ℝ2\{b\}\_\{o\}\\in\\mathbb\{R\}^\{2\}are the learnable weight matrix and bias vector of the binary classifier, respectively\.y^∈\[0,1\]\\widehat\{y\}\\in\[0,1\]denotes the predicted probability of in\-hospital mortality\.

## 4Experiment

### 4\.1Datasets

We evaluate our method on multimodal ICU cohorts from MIMIC\-III\[[16](https://arxiv.org/html/2608.21941#bib.bib24)\]and MIMIC\-IV v3\.1\.111[https://physionet\.org/content/mimiciv/3\.1/](https://physionet.org/content/mimiciv/3.1/)For MIMIC\-III, we follow the processed irregular\-EHR benchmark cohort of Zhang et al\.\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]; for MIMIC\-IV, clinical notes are taken from MIMIC\-IV\-Note\.222[https://physionet\.org/content/mimic\-iv\-note/2\.2/](https://physionet.org/content/mimic-iv-note/2.2/)The task is 48\-hour in\-hospital mortality prediction \(48\-IHM\), where each sample uses the first 48 hours after ICU admission to predict hospital mortality\. It is a binary classification problem with label imbalance with death to discharge ratio of approximately 1:7\. ICU stays shorter than 48 hours are excluded\.

Each sample contains a structured MISTS modality and a clinical\-note modality\. MISTS includes 17 benchmark clinical variables with irregular timestamps and observation masks; we also build a one\-hour value\-and\-mask representation and normalize continuous variables using training\-set statistics\. Clinical notes are aligned by ICU admission time, and at most the latest five notes within the 48\-hour window are tokenized with Clinical\-Longformer and truncated or padded to 1024 tokens per note\.

### 4\.2Evaluation Metrics

Following prior work\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\], we evaluate 48\-IHM as an imbalanced binary prediction task in which identifying mortality cases is clinically important\. We report F1 to characterize classification performance at a fixed decision threshold, AUPRC to evaluate positive\-case retrieval under label imbalance, and AUROC to provide a complementary measure of overall discrimination\. Because the dataset is class\-imbalanced and mortality is the minority class, we use AUPRC as the primary metric for model selection, as it better reflects the model’s ability to identify mortality cases\.

### 4\.3Baselines

To ensure a comprehensive comparison, we evaluate our method against three groups of baselines: MISTS\-only approaches, Notes\-only approaches, and Full\-modality approaches\. For MIMIC\-III, we follow the published irregular\-EHR baseline results\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]; for MIMIC\-IV, we evaluate the listed baselines under our data split\.

For the*MISTS\-only*setting on MIMIC\-III, we follow the benchmark comparison with Imputation\[[22](https://arxiv.org/html/2608.21941#bib.bib42)\], IP\-Net\[[31](https://arxiv.org/html/2608.21941#bib.bib25)\], mTAND\[[32](https://arxiv.org/html/2608.21941#bib.bib23)\], GRU\-D\[[7](https://arxiv.org/html/2608.21941#bib.bib26)\], RAINDROP\[[45](https://arxiv.org/html/2608.21941#bib.bib27)\], and UTDE\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]\. On MIMIC\-IV, we evaluate the regular time\-series backbones IP\-Net, GRU\-D, SeFT\[[13](https://arxiv.org/html/2608.21941#bib.bib44)\], DGM2\-O\[[38](https://arxiv.org/html/2608.21941#bib.bib45)\], mTAND, RAINDROP, MTGNN\[[40](https://arxiv.org/html/2608.21941#bib.bib36)\], and UTDE\.

For the*Notes\-only*setting on both datasets, we compare with the note\-based models actually used in our experiments: Flat\[[9](https://arxiv.org/html/2608.21941#bib.bib28)\], HierTrans\[[26](https://arxiv.org/html/2608.21941#bib.bib29)\], T\-LSTM\[[6](https://arxiv.org/html/2608.21941#bib.bib30)\], FT\-LSTM\[[44](https://arxiv.org/html/2608.21941#bib.bib31)\], GRU\-D\[[7](https://arxiv.org/html/2608.21941#bib.bib26)\], and mTAND\-txt\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\]\.

For the*Full\-modality*setting on MIMIC\-IV, we compare with representative multimodal EHR and feature\-fusion baselines, including MMIMIC\[[46](https://arxiv.org/html/2608.21941#bib.bib20)\], CTPD\[[36](https://arxiv.org/html/2608.21941#bib.bib35)\], MedFuse\[[12](https://arxiv.org/html/2608.21941#bib.bib37)\], DAFT\[[28](https://arxiv.org/html/2608.21941#bib.bib38)\], MMTM\[[17](https://arxiv.org/html/2608.21941#bib.bib39)\], and DrFuse\[[43](https://arxiv.org/html/2608.21941#bib.bib40)\]\. Under each missing\-modality setting, our model receives only the available modality and reconstructs the missing stream through prompt\-based cross\-modal generation\.

### 4\.4Implementation Details

#### Architecture and Training Details\.

We use a shared embedding dimension ofd=128d=128and a unified timeline of lengthT=48T=48\. Clinical notes are pre\-tokenized by Clinical\-Longformer with a maximum token length of 1024, and are temporally aligned to the unified timeline using mTAND\-txt\. For the MISTS stream, we adopt UTDE, which incorporates a learnable gate based on mTAND\-ts\. The text encoder is initialized from a pretrained Clinical\-Longformer checkpoint, while the remaining task\-specific modules are trained from scratch\. Each run is trained up to a fixed maximum number of epochs, and model selection uses validation AUPRC under the corresponding missing\-modality scenario\.

#### Prompt initialization\.

Generative prompt queries are initialized from the training\-set global means of the projected target\-modality embeddings\. The modality\-signal and missing\-type prompts are initialized to zero, whereas the missing\-mode\-specific temporal prompts are independently initialized from a standard normal distribution\. All prompt parameters are jointly optimized with the remaining task\-specific modules\.

#### Training\-Time Modality Dropout\.

To expose the model to incomplete inputs during training, we apply modality\-level dropout to the paired multimodal training samples\. For each complete training batch, we sample a Bernoulli variable with probabilityη\\eta\. If dropout is not activated, both MISTS and clinical notes are kept and the sample is trained under the complete casem=cm=\\mathrm\{c\}\. If dropout is activated, one modality is removed at the input level according to the target missing scenario\.

### 4\.5Main Results

We first examine our method’s robustness when either modality is unavailable, followed by assessing whether the proposed framework retains competitive performance when both modalities are observed\.

Table 1:Main results on MIMIC\-III under single\-modality settings\. Best results are shown in bold and second\-best results are underlined\.Table 2:Main results on MIMIC\-IV under single\-modality settings\. Best results within each setting and metric are shown in bold and second\-best results are underlined\.
#### Missing\-modality Comparison

To examine whether the proposed framework can recover useful information beyond that captured by unimodal models, Tables[1](https://arxiv.org/html/2608.21941#S4.T1)and[2](https://arxiv.org/html/2608.21941#S4.T2)compare our method with strong MISTS\-only and Notes\-only baselines on two independent cohorts\. Our method achieves the highest AUPRC in all four settings\. Since mortality cases are underrepresented and AUPRC is our primary metric, this consistent result provides strong evidence that our prompt\-based framework improves the identification and ranking of high\-risk patients under missing modalities\. Moreover, the gains are more consistent across metrics in the Notes\-only setting than in the MISTS\-only setting\. This asymmetry suggests that reconstructing MISTS from clinical notes is generally more stable than reconstructing clinical notes from MISTS\.

Table 3:Full\-modality results on MIMIC\-IV, where both MISTS and clinical notes are available at test time\. Best results are shown in bold and second\-best results are underlined\.
#### Full\-modality Comparison\.

We further evaluate whether our method maintains its performance when both modalities are available\. As shown in Table[3](https://arxiv.org/html/2608.21941#S4.T3), our method achieves the best F1 and the second\-best AUROC\. Compared with MMIMIC, which serves as our backbone, it substantially improves F1, with only modest decreases in AUROC and AUPRC\. These results indicate that the proposed method largely preserves the backbone’s predictive capability under complete observations in addition to handling missing modalities effectively\.

### 4\.6Ablation Study

To better understand the contribution of each design choice, we conduct ablation studies from two perspectives: contribution of different prompts and training\-time modality dropout rate\.

Table 4:Prompt ablation under the Notes\-only setting on MIMIC\-III, including individual, three\-prompt, and cumulative configurations\. A check mark indicates that the corresponding prompt is enabled\. Best and second\-best results are shown in bold and underlined, respectively\.Prompt ConfigurationPerformancePGP\_\{G\}PSP\_\{S\}PTP\_\{T\}PτP\_\{\\tau\}AUPRCAUROC0\.53830\.8759✓\\checkmark0\.55660\.8841✓\\checkmark0\.55720\.8839✓\\checkmark0\.55270\.8822✓\\checkmark0\.54120\.8782✓\\checkmark✓\\checkmark0\.56950\.8864✓\\checkmark✓\\checkmark✓\\checkmark0\.57680\.8917✓\\checkmark✓\\checkmark✓\\checkmark✓\\checkmark0\.58530\.8904✓\\checkmark✓\\checkmark✓\\checkmark0\.55970\.8874✓\\checkmark✓\\checkmark✓\\checkmark0\.57000\.8872✓\\checkmark✓\\checkmark✓\\checkmark0\.56870\.8875#### Prompt Ablation\.

Table[4](https://arxiv.org/html/2608.21941#S4.T4)demonstrates the effectiveness and complementarity of the four prompts\. Enabling any single prompt yields modest improvements in both AUPRC and AUROC, indicating that each provides useful task\-specific information\. The cumulative configurations show a clear progression:PGP\_\{G\}reconstructs the missing modality,PSP\_\{S\}identifies the provenance of generated and observed features,PTP\_\{T\}adapts cross\-modal interaction to the modality\-availability pattern, andPτP\_\{\\tau\}performs condition\-specific temporal aggregation\. As these functions are progressively integrated, AUPRC increases steadily and reaches its highest value of 0\.5853 with all prompts enabled\. Consistently, removing any prompt from the complete configuration reduces AUPRC to varying degrees, supporting that the four prompts make non\-redundant contributions and work together to improve missing\-modality prediction\.

Figure 3:Modality dropout rate ablation on MIMIC\-III\. Each panel reports AUPRC on the left y\-axis and AUROC on the right y\-axis forη∈\{0\.1,0\.3,0\.5,0\.7,0\.9\}\\eta\\in\\\{0\.1,0\.3,0\.5,0\.7,0\.9\\\}\.
#### Dropout Rate\.

We vary the training\-time modality dropout rateη\\etato examine how exposure to simulated missing inputs affects robustness\. As shown in Figure[3](https://arxiv.org/html/2608.21941#S4.F3), both settings show a non\-monotonic trend\. When the dropout rate is too low, the model is not sufficiently exposed to incomplete cases during training; when it is too high, effective observed supervision is reduced and the generated representations become less reliable\. These observations suggest that a moderate\-to\-high dropout rate forces the model to learn stronger missing\-modality compensation and more robust cross\-modal interaction under incomplete observations\.

#### Representation Visualization\.

![Refer to caption](https://arxiv.org/html/2608.21941v1/completion_tsne_grid_onecol.png)Figure 4:t\-SNE visualization of classifier\-level representations on MIMIC\-III\. Panel \(a\) and \(b\) show the MISTS\-only and Notes\-only settings respectively\. Each panel compares the observed single\-modality representation, its prompt\-completed representation, and the fully observed MISTS\+Notes reference\.To examine whether prompt completion recovers meaningful cross\-modal structure, we visualize the learned representations on MIMIC\-III\. As shown in Figure[4](https://arxiv.org/html/2608.21941#S4.F4), the completed representations shift from the single\-modality region toward the Full\-modality region while retaining the structure of the observed modality\. This controlled shift indicates that prompt completion supplements rather than overwrites the available evidence, reducing the representation gap between incomplete and complete EHR inputs\.

Figure 5:Evolution of the predicted mortality risk for a representative patient in the MIMIC\-III Notes\-only setting as the prompt components are cumulatively activated\.All\-offindicates that no prompts are used\. Predictions above the decision threshold are classified as mortality, whereas those below the threshold are classified as survival\.![Refer to caption](https://arxiv.org/html/2608.21941v1/case_study_notes.png)Figure 6:Changes in predicted mortality risk after removing each note for the same patient\. Negative scores shown in green indicate evidence supporting survival, whereas positive scores shown in red indicate evidence supporting mortality\.

### 4\.7Case Study

To examine how the prompts affect individual predictions and how the model integrates evidence across clinical notes, we conduct a case study on a representative patient\. Figure[5](https://arxiv.org/html/2608.21941#S4.F5)illustrates how the prompts correct a survival case under the MIMIC\-III Notes\-only setting\. Without prompts, the model incorrectly predicts mortality\.PGP\_\{G\}produces the largest risk reduction, showing that the reconstructed MISTS stream supplies the main complementary evidence\.PSP\_\{S\}then moves the prediction below the decision threshold, whilePTP\_\{T\}andPτP\_\{\\tau\}maintain the correct decision\. This progression indicates that reconstruction drives the correction, whereas the remaining prompts refine how generated and observed evidence is interpreted\.

Figure[6](https://arxiv.org/html/2608.21941#S4.F6)further shows that the prediction is based on evidence distributed across multiple notes\. The impaired responsiveness in N2 and severe hypoglycemia in N4 support mortality, whereas N1 and N5 contain evidence favoring survival\. The model therefore reaches the correct prediction by balancing conflicting clinical observations rather than relying on a single note\.

## 5Conclusion

In this paper, we present a unified multimodal prompt\-learning framework for robust mortality prediction from irregular and incomplete EHRs\. We proposed generative prompts, missing\-signal prompts, missing\-type prompts, and temporal prompts to capture information under missing\-modality conditions\. Ablation study and case study further confirm that the four prompts play complementary roles in aligning incomplete inputs with the full\-modality decision space\. Future work can evaluate the framework on broader clinical prediction tasks and datasets, and extend it to settings involving more modalities and complex real\-world missingness patterns\.

## References

- \[1\]J\. Adler\-Milstein, C\. M\. DesRoches, P\. Kralovec, G\. Foster, C\. Worzala, D\. Charles, T\. Searcy, and A\. K\. Jha\(2015\)Electronic health record adoption in us hospitals: progress continues, but challenges persist\.Health affairs34\(12\),pp\. 2174–2180\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[2\]B\. Afessa, O\. Gajic, and M\. T\. Keegan\(2007\)Severity of illness and organ failure assessment in adult intensive care units\.Critical care clinics23\(3\),pp\. 639–658\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[3\]S\. F\. Ahmed, M\. S\. B\. Alam, M\. Hassan, M\. R\. Rozbu, T\. Ishtiak, N\. Rafa, M\. Mofijur, A\. Shawkat Ali, and A\. H\. Gandomi\(2023\)Deep learning modelling techniques: current progress, applications, advantages, and challenges\.Artificial Intelligence Review56\(11\),pp\. 13521–13617\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[4\]C\. Alberti, C\. Brun\-Buisson, H\. Burchardi, C\. Martin, S\. Goodman, A\. Artigas, A\. Sicignano, M\. Palazzo, R\. Moreno, R\. Boulmé,et al\.\(2002\)Epidemiology of sepsis and infection in icu patients from an international multicentre cohort study\.Intensive care medicine28,pp\. 108–121\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[5\]H\. Atasoy, B\. N\. Greenwood, and J\. S\. McCullough\(2019\)The digitization of patient care: a review of the effects of electronic health records on health care quality and utilization\.Annual review of public health40,pp\. 487–500\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[6\]I\. M\. Baytas, C\. Xiao, X\. Zhang, F\. Wang, A\. K\. Jain, and J\. Zhou\(2017\)Patient subtyping via time\-aware LSTM networks\.InProceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining \(KDD\),pp\. 65–74\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1)\.
- \[7\]Z\. Che, S\. Purushotham, K\. Cho, D\. Sontag, and Y\. Liu\(2018\)Recurrent neural networks for multivariate time series with missing values\.Scientific Reports8\(1\),pp\. 6085\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1)\.
- \[8\]D\. J\. Cullen, B\. J\. Sweitzer, D\. W\. Bates, E\. Burdick, A\. Edmondson, and L\. L\. Leape\(1997\)Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units\.Critical care medicine25\(8\),pp\. 1289–1297\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[9\]I\. Deznabi, M\. Iyyer, and M\. Fiterau\(2021\)Predicting in\-hospital mortality by combining clinical notes with time\-series data\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,pp\. 4026–4031\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1)\.
- \[10\]T\. Gao, A\. Fisch, and D\. Chen\(2021\)Making pre\-trained language models better few\-shot learners\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 3816–3830\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p3.1),[§2](https://arxiv.org/html/2608.21941#S2.p3.1)\.
- \[11\]Z\. Guo, T\. Jin, and Z\. Zhao\(2024\)Multimodal prompt learning with missing modalities for sentiment analysis and emotion recognition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1726–1736\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p3.1)\.
- \[12\]N\. Hayat, K\. J\. Geras, and F\. E\. Shamout\(2022\)MedFuse: multi\-modal fusion with clinical time\-series data and chest x\-ray images\.InProceedings of the 7th Machine Learning for Healthcare Conference,Proceedings of Machine Learning Research, Vol\.182,pp\. 479–503\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[13\]M\. Horn, M\. Moor, C\. Bock, B\. Rieck, and K\. Borgwardt\(2020\)Set functions for time series\.InProceedings of the 37th International Conference on Machine Learning,pp\. 4353–4363\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[14\]T\. Hourani, A\. Perez\-Gonzalez, K\. Khoshmanesh, R\. Luwor, A\. A\. Achuthan, S\. Baratchi, N\. M\. O’Brien\-Simpson, and A\. Al\-Hourani\(2023\)Label\-free macrophage phenotype classification using machine learning methods\.Scientific Reports13\(1\),pp\. 5202\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[15\]A\. E\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow,et al\.\(2023\)MIMIC\-iv, a freely accessible electronic health record dataset\.Scientific data10\(1\),pp\. 1\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p4.1)\.
- \[16\]A\. E\. Johnson, T\. J\. Pollard, L\. Shen, L\. H\. Lehman, M\. Feng, M\. Ghassemi, B\. Moody, P\. Szolovits, L\. Anthony Celi, and R\. G\. Mark\(2016\)MIMIC\-iii, a freely accessible critical care database\.Scientific data3\(1\),pp\. 1–9\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p4.1),[§4\.1](https://arxiv.org/html/2608.21941#S4.SS1.p1.1)\.
- \[17\]H\. R\. V\. Joze, A\. Shaban, M\. L\. Iuzzolino, and K\. Koishida\(2020\)MMTM: multimodal transfer module for CNN fusion\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 13289–13299\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[18\]M\. U\. Khattak, H\. Rasheed, M\. Maaz, S\. Khan, and F\. S\. Khan\(2023\)Maple: multi\-modal prompt learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 19113–19122\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p3.1),[§2](https://arxiv.org/html/2608.21941#S2.p3.1)\.
- \[19\]Y\. Lee, Y\. Tsai, W\. Chiu, and C\. Lee\(2023\)Multimodal prompting with missing modalities for visual recognition\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14943–14952\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1),[§2](https://arxiv.org/html/2608.21941#S2.p3.1)\.
- \[20\]Y\. Li, R\. M\. Wehbe, F\. S\. Ahmad, H\. Wang, and Y\. Luo\(2022\)Clinical\-Longformer and clinical\-BigBird: transformers for long clinical sequences\.Journal of Biomedical Informatics130,pp\. 104069\.External Links:[Document](https://dx.doi.org/10.1016/j.jbi.2022.104069)Cited by:[§3\.2](https://arxiv.org/html/2608.21941#S3.SS2.p1.1)\.
- \[21\]Z\. Liang, Z\. Pan, and R\. Xiong\(2025\)Causal representation learning from multimodal clinical records under non\-random modality missingness\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 28779–28796\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.
- \[22\]Z\. C\. Lipton, D\. Kale, and R\. Wetzel\(2016\)Directly modeling missing data in sequences with rnns: improved classification of clinical time series\.InMachine learning for healthcare conference,pp\. 253–270\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[23\]S\. Liu and H\. Chen\(2024\)Knowledge injected multimodal irregular ehrs model for medical prediction\.InInternational Workshop on Trustworthy Artificial Intelligence for Healthcare,pp\. 25–39\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p2.1),[§2](https://arxiv.org/html/2608.21941#S2.p1.1)\.
- \[24\]S\. Liu, X\. Wang, J\. Du, Y\. Hou, X\. Zhao, H\. Xu, H\. Wang, Y\. Xiang, and B\. Tang\(2023\)SHAPE: a sample\-adaptive hierarchical prediction network for medication recommendation\.IEEE Journal of Biomedical and Health Informatics27\(12\),pp\. 6018–6028\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[25\]M\. J\. Otero\-López, P\. Alonso\-Hernández, J\. A\. Maderuelo\-Fernández, B\. Garrido\-Corro, A\. Domínguez\-Gil, and A\. Sánchez\-Rodríguez\(2006\)Preventable adverse drug events in hospitalized patients\.Medicina clinica126\(3\),pp\. 81–87\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[26\]R\. Pappagari, P\. Żelasko, J\. Villalba, Y\. Carmiel, and N\. Dehak\(2019\)Hierarchical transformers for long document classification\.InIEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\),pp\. 838–844\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1)\.
- \[27\]H\. Pham, P\. P\. Liang, T\. Manzini, L\. Morency, and B\. Póczos\(2019\)Found in translation: learning robust joint representations by cyclic translations between modalities\.InProceedings of the AAAI conference on artificial intelligence,Vol\.33,pp\. 6892–6899\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.
- \[28\]S\. Pölsterl, T\. N\. Wolf, and C\. Wachinger\(2021\)Combining 3d image and tabular data via the dynamic affine feature map transform\.InMedical Image Computing and Computer Assisted Intervention – MICCAI 2021,Lecture Notes in Computer Science, Vol\.12905,pp\. 688–698\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-87240-3%5F66)Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[29\]A\. Rajkomar, E\. Oren, K\. Chen, A\. M\. Dai, N\. Hajaj, M\. Hardt, P\. J\. Liu, X\. Liu, J\. Marcus, M\. Sun,et al\.\(2018\)Scalable and accurate deep learning with electronic health records\.NPJ digital medicine1\(1\),pp\. 1–10\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[30\]B\. Shickel, P\. J\. Tighe, A\. Bihorac, and P\. Rashidi\(2017\)Deep ehr: a survey of recent advances in deep learning techniques for electronic health record \(ehr\) analysis\.IEEE journal of biomedical and health informatics22\(5\),pp\. 1589–1604\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[31\]S\. N\. Shukla and B\. M\. Marlin\(2019\)Interpolation\-prediction networks for irregularly sampled time series\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[32\]S\. N\. Shukla and B\. M\. Marlin\(2021\)Multi\-time attention networks for irregularly sampled time series\.External Links:2101\.10318Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[33\]S\. A\. Tisherman and D\. M\. Stein\(2018\)ICU management of trauma patients\.Critical Care Medicine46\(12\),pp\. 1991–1997\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[34\]J\. K\. Valik, L\. Ward, H\. Tanushi, A\. F\. Johansson, A\. Färnert, M\. L\. Mogensen, B\. W\. Pickering, V\. Herasevich, H\. Dalianis, A\. Henriksson,et al\.\(2023\)Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data\.Scientific reports13\(1\),pp\. 11760\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[35\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§3\.2](https://arxiv.org/html/2608.21941#S3.SS2.p4.1)\.
- \[36\]F\. Wang, F\. Wu, Y\. Tang, and L\. Yu\(2025\)CTPD: cross\-modal temporal pattern discovery for enhanced multimodal electronic health records analysis\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 6783–6799\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.352)Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[37\]M\. Wang, S\. Fan, Y\. Li, Z\. Xie, and H\. Chen\(2025\)Missing\-modality enabled multi\-modal fusion architecture for medical data\.Journal of Biomedical Informatics164,pp\. 104796\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.
- \[38\]Y\. Wu, J\. Ni, W\. Cheng, B\. Zong, D\. Song, Z\. Chen, Y\. Liu, X\. Zhang, H\. Chen, and S\. B\. Davidson\(2021\)Dynamic gaussian mixture based deep generative model for robust forecasting on sparse multivariate time series\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 651–659\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[39\]Z\. Wu, A\. Dadu, N\. Tustison, B\. Avants, M\. Nalls, J\. Sun, and F\. Faghri\(2024\)Multimodal patient representation learning with missing modalities and labels\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 55538–55557\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.
- \[40\]Z\. Wu, S\. Pan, G\. Long, J\. Jiang, X\. Chang, and C\. Zhang\(2020\)Connecting the dots: multivariate time series forecasting with graph neural networks\.InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,pp\. 753–763\.External Links:[Document](https://dx.doi.org/10.1145/3394486.3403118)Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[41\]C\. Xiao, E\. Choi, and J\. Sun\(2018\)Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review\.Journal of the American Medical Informatics Association25\(10\),pp\. 1419–1428\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[42\]F\. Xie, B\. Chakraborty, M\. E\. H\. Ong, B\. A\. Goldstein, N\. Liu,et al\.\(2020\)Autoscore: a machine learning–based automatic clinical score generator and its application to mortality prediction using electronic health records\.JMIR medical informatics8\(10\),pp\. e21798\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[43\]W\. Yao, K\. Yin, W\. K\. Cheung, J\. Liu, and J\. Qin\(2024\)Drfuse: learning disentangled representation for clinical multi\-modal fusion with missing modality and modal inconsistency\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 16416–16424\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[44\]D\. Zhang, J\. Thadajarassiri, C\. Sen, and E\. Rundensteiner\(2020\)Time\-aware transformer\-based network for clinical notes series prediction\.InMachine learning for healthcare conference,pp\. 566–588\.Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1)\.
- \[45\]X\. Zhang, M\. Zeman, T\. Tsiligkaridis, and M\. Zitnik\(2022\)Graph\-guided network for irregularly sampled multivariate time series\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1)\.
- \[46\]X\. Zhang, S\. Li, Z\. Chen, X\. Yan, and L\. R\. Petzold\(2023\)Improving medical predictions by irregular multimodal electronic health records modeling\.InInternational Conference on Machine Learning,pp\. 41300–41313\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p2.1),[§2](https://arxiv.org/html/2608.21941#S2.p1.1),[§3\.2](https://arxiv.org/html/2608.21941#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.21941#S4.SS1.p1.1),[§4\.2](https://arxiv.org/html/2608.21941#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p2.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2608.21941#S4.SS3.p4.1)\.
- \[47\]Y\. Zhang, X\. Wu, Q\. Fang, S\. Qian, and C\. Xu\(2023\)Knowledge\-enhanced attributed multi\-task learning for medicine recommendation\.ACM Transactions on Information Systems41\(1\),pp\. 1–24\.Cited by:[§1](https://arxiv.org/html/2608.21941#S1.p1.1)\.
- \[48\]J\. Zhao, R\. Li, and Q\. Jin\(2021\)Missing modality imagination network for emotion recognition with uncertain missing modalities\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 2608–2618\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.203)Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.
- \[49\]Y\. Zhao, C\. Chen, Q\. Y\. Pang, Y\. Fu, Q\. Li, C\. Tang, B\. T\. Ang, and Y\. Jin\(2026\)Tackling dual\-stage missing modalities in brain tumor segmentation via robust modality reconstruction and prompt\-guided modality adaptation\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 13314–13322\.Cited by:[§2](https://arxiv.org/html/2608.21941#S2.p2.1)\.