MultiSigBERT: Beyond Survival Analysis through Multimodal and Sequential Modeling in Oncology

arXiv cs.LG Papers

Summary

MultiSigBERT is a novel multimodal framework for survival analysis in oncology that uses path signature transforms to integrate electronic health record data for improved risk prediction.

arXiv:2608.16972v1 Announce Type: new Abstract: Machine learning has become an essential component of modern healthcare, where the integration of heterogeneous data sources offers unprecedented opportunities to improve clinical decision-making. Electronic Health Records (EHR) contain complementary information -- including narrative clinical reports, numerical measurements, and structured variables -- yet most survival models remain limited to a single modality or fail to exploit the temporal nature of patient trajectories. We propose MultiSigBERT, a unified framework for multimodal sequential survival modeling in oncology based on path signature representations. Here, narrative medical reports (free-text) are converted into sentence embeddings by extracting and averaging contextual word embeddings. These representations are then compressed via modality-specific PCA and concatenated with structured covariates to form joint temporal trajectories which are then encoded using the Signature transform, a tool from Rough Paths theory that efficiently captures higher-order temporal interactions across modalities without supervision needed. The computed Signature features are finally incorporated as high dimensional features into a LASSO-regularized Cox model to estimate individualized risk scores. The performance of our novel MultiSigBERT pipeline is illustrated on the analysis of a real-world oncology cohort from the L\'eon B\'erard Center, comprising over 120,000 medical reports and structured records from more than 2,500 patients. The model achieves a concordance index of 0.743 (sd 0.029) on an independent test set, demonstrating the benefit of jointly modeling multimodal temporal dynamics together with patient-level geometric structure for survival prediction.
Original Article
View Cached Full Text

Cached at: 08/19/26, 10:19 AM

# MultiSigBERT: Beyond Survival Analysis through Multimodal and Sequential Modeling in Oncology
Source: [https://arxiv.org/html/2608.16972](https://arxiv.org/html/2608.16972)
Paul Minchella \(✉\)Affiliation:Université Lumière Lyon 2, 69500 Bron, FranceE\-mail[\{paul\.minchella, stephane\.chretien, guillaume\.metzler\}@univ\-lyon2\.fr](mailto:{paul.minchella,%20stephane.chretien,%20guillaume.metzler}@univ-lyon2.fr)Stéphane ChrétienAffiliation:Université Lumière Lyon 2, 69500 Bron, FranceE\-mail[\{paul\.minchella, stephane\.chretien, guillaume\.metzler\}@univ\-lyon2\.fr](mailto:{paul.minchella,%20stephane.chretien,%20guillaume.metzler}@univ-lyon2.fr)Guillaume MetzlerAffiliation:Université Lumière Lyon 2, 69500 Bron, FranceE\-mail[\{paul\.minchella, stephane\.chretien, guillaume\.metzler\}@univ\-lyon2\.fr](mailto:{paul.minchella,%20stephane.chretien,%20guillaume.metzler}@univ-lyon2.fr)Loic VerlingueRémi VaucherAffiliation:EPITA, Lyon 69007, FranceE\-mail[remi\.vaucher@epita\.fr](mailto:[email protected])

###### Abstract

Machine learning has become an essential component of modern healthcare, where the integration of heterogeneous data sources offers unprecedented opportunities to improve clinical decision\-making\. Electronic Health Records \(EHR\) contain complementary information – including narrative clinical reports, numerical measurements, and structured variables – yet most survival models remain limited to a single modality or fail to exploit the temporal nature of patient trajectories\. We propose MultiSigBERT, a unified framework for multimodal sequential survival modeling in oncology based on path signature representations\. Here, narrative medical reports \(free\-text\) are converted into sentence embeddings by extracting and averaging contextual word embeddings\. These representations are then compressed via modality\-specific PCA and concatenated with structured covariates to form joint temporal trajectories which are then encoded using the Signature transform, a tool from Rough Paths theory that efficiently captures higher\-order temporal interactions across modalities without supervision needed\. The computed Signature features are finally incorporated as high dimensional features into a LASSO\-regularized Cox model to estimate individualized risk scores\.

The performance of our novel MultiSigBERT pipeline is illustrated on the analysis of a real\-world oncology cohort from the Léon Bérard Center, comprising over 120,000 medical reports and structured records from more than 2,500 patients\. The model achieves a concordance index of 0\.743 \(sd 0\.029\)\. on an independent test set, demonstrating the benefit of jointly modeling multimodal temporal dynamics together with patient\-level geometric structure for survival prediction\.

###### Keywords:

Multimodal NLP Signature Transform Survival Analysis Oncology\.

## 1Introduction

### 1\.1Background and Challenges

Survival analysis aims to model the time until an event of interest occurs, such as death, recurrence, or disease progression\. In oncology, accurate survival prediction is essential for treatment selection, clinical trial inclusion, and follow\-up planning\.

Modern electronic health records \(EHRs\) contain heterogeneous and complementary information, including structured clinical variables, laboratory measurements, molecular data, and narrative medical reports\. This diversity offers strong potential to improve survival modeling\. However, integrating these data sources remains challenging\. Most classical survival models rely on static baseline covariates and ignore the longitudinal nature of patient follow\-up\. Even recent machine learning approaches often struggle with irregular, asynchronous, and time\-dependent clinical data\.

In addition, many existing methods are unimodal, focusing either on structured variables or unstructured text\. Such approaches fail to capture the interactions between modalities that shape patient trajectories and influence prognosis\.

To address these limitations, we introduce MultiSigBERT, a multimodal and sequential survival framework designed to integrate heterogeneous clinical data while modeling temporal dynamics in a principled and interpretable manner\. By combining embedding representations, rough path signatures, graph\-based clustering, and sparse survival modeling, this model provides a unified approach for analyzing real\-world multimodal oncology data\.

### 1\.2Related Works

Survival analysis is a fundamental task in clinical research, aiming to estimate the time to an event such as death or relapse\. The Cox Proportional Hazards model\[[7](https://arxiv.org/html/2608.16972#bib.bib7)\]remains the classical reference due to its interpretability and solid theoretical basis\. Regularized extensions, notably LASSO penalization\[[24](https://arxiv.org/html/2608.16972#bib.bib24)\], have enabled its application to high\-dimensional settings\. More recently, neural\-network\-based survival models have been proposed to capture non\-linear effects and complex feature interactions\. DeepSurv\[[15](https://arxiv.org/html/2608.16972#bib.bib15)\]adapted the Cox partial likelihood to deep architectures for personalized prediction from structured static data\. To model longitudinal dynamics, Dynamic\-DeepHit\[[17](https://arxiv.org/html/2608.16972#bib.bib17)\]introduced recurrent networks with temporal attention\. DySurv\[[20](https://arxiv.org/html/2608.16972#bib.bib20)\]further incorporated conditional variational autoencoders to integrate time\-dependent EHR features\.

From a mathematical perspective, rough path theory has recently emerged as a principled framework for encoding temporal data\. CoxSig\[[3](https://arxiv.org/html/2608.16972#bib.bib3)\]introduced signature transforms into survival modeling, demonstrating strong empirical performance together with theoretical guarantees such as universality and invariance to time reparameterization\.

The authors of SigBERT\[[21](https://arxiv.org/html/2608.16972#bib.bib21)\]introduced a framework combining a domain\-specific language model with rough path signature features for temporal survival analysis in oncology\. Applied to narrative reports from the Léon Bérard Center, SigBERT achieved a concordance index of 0\.75 \(sd 0\.014\) and a time\-dependent AUC of 0\.80 \(sd 0\.029\), demonstrating that properly encoded clinical text can substantially improve survival risk estimation\.

However, SigBERT focuses exclusively on narrative medical reports and does not explicitly address the integration of additional clinical modalities\. More broadly, the principled combination of heterogeneous clinical data sources remains a major challenge in survival modeling\. A summary of the related methods is provided in the first table of the Supplementary Material\.

### 1\.3Our Contributions

Building upon SigBERT\[[21](https://arxiv.org/html/2608.16972#bib.bib21)\], which focused on narrative medical reports, we extend this approach to a multimodal setting\. Moreover, the original framework was not formulated within a landmark prediction design, which may allow temporal information leakage when constructing temporal features\.

To address these limitations, we introduce , a multimodal survival modeling framework designed to integrate heterogeneous clinical data in oncology\. The proposed approach jointly leverages unstructured clinical text, structured numerical variables, and their longitudinal evolution within a unified temporal representation\. More precisely, this work makes three main contributions\.

First, we introduce a multimodal temporal encoding framework that combines textual embeddings and structured clinical variables into synchronized trajectories prior to signature transformation\. Narrative reports are first encoded into sentence embeddings using OncoBERT\[[26](https://arxiv.org/html/2608.16972#bib.bib26)\]\. Structured numerical variables are processed separately and aligned with the same temporal axis, thanks to linear interpolation\. The two modalities are then concatenated at each time point to form multimodal vectors, which are subsequently encoded using the signature transform, enabling the model to capture higher\-order temporal dependencies and cross\-modal interactions in a mathematically principled way\.

Second, we propose a robust landmark\-based survival modeling pipeline that prevents temporal information leakage by restricting feature construction to observations available before the prediction time, thereby enabling well\-defined dynamic risk prediction from longitudinal clinical data\.

Finally, we provide an efficient and interpretable survival modeling framework combining signature features with sparse Cox regression\. This design enables scalable analysis of large clinical cohorts while maintaining interpretability of the predictive factors\. In addition, we conduct a systematic empirical comparison of modality\-specific contributions by evaluating text\-only fully multimodal configurations, thereby quantifying the added value of multimodal fusion for survival prediction\.

To our knowledge, is the first survival framework that jointly combines multimodal temporal encoding of textual and structured numerical data with sparse survival modeling within a coherent and computationally efficient pipeline\.

The complete implementation is available in the associated GitHub repository at[https://github\.com/MINCHELLA\-Paul/MultiSigBERT](https://github.com/MINCHELLA-Paul/MultiSigBERT)\. The main notebookmultisigbert\_study\.ipynbcontains additional experimental results, and all descriptive statistics figures are provided in`results/descriptive\_statistics`\.

## 2Method

### 2\.1Global overview

The proposed MultiSigBERT framework integrates heterogeneous clinical data into a unified temporal representation for survival modeling\. The dataset consists of longitudinal patient\-level records combining narrative medical reports, structured time\-dependent clinical variables, and right\-censored survival outcomes\. Formally, for each patientii, we observe a sequence of clinical time points

\{t1\(i\),…,tNi\(i\)\},\\left\\\{t^\{\(i\)\}\_\{1\},\\dots,t^\{\(i\)\}\_\{\{N\_\{i\}\}\}\\right\\\},at which textual reports and structured covariates are recorded\. Each report is encoded as a sentence embedding, yielding a time\-indexed representation of the patient’s clinical trajectory\. In addition, we observe structured longitudinal variables measured at irregular time points and aligned with the same temporal axis\.

The survival outcome is defined by a pair\(Ti,Di\)\(T\_\{i\},D\_\{i\}\), whereTi≥0T\_\{i\}\\geq 0denotes the observed time\-to\-event or censoring, andDi∈\{0,1\}D\_\{i\}\\in\\\{0,1\\\}is the event indicator, withDi=1D\_\{i\}=1if the event \(e\.g\., death\) is observed andDi=0D\_\{i\}=0if the observation is right\-censored\. The objective is to learn a mapping from the patient\-specific temporal representation to a risk function compatible with the Cox proportional hazards framework\.

For each patient, clinical notes are observed at successive timestamps, forming an ordered sequence of unstructured text that reflects disease progression, treatment response, and medical decisions over time\. In our cohort, these documents consist mainly of consultation reports \(68%\) and hospitalization reports \(27%\), offering a rich temporal source of prognostic information\. In parallel, structured variables such asWeight,Karnofsky Index,Blood Pressure, andPulse Rateare recorded at irregular timepoints\. These quantitative measures provide complementary information about the patient’s physiological state\. All modalities are preprocessed and temporally aligned to construct coherent multimodal trajectories\. Detailed descriptive statistics are reported in the notebook`descriptive\_statistics\.ipynb`provided in the associated GitHub repository\.

To prevent information leakage and define a consistent prediction task, we adopt a landmark design\. For a fixed landmark time𝐋\\boldsymbol\{\\mathrm\{L\}\}, only data observed within the backward window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]are used to construct the patient representation, and survival is predicted beyond𝐋\\boldsymbol\{\\mathrm\{L\}\}\. This framework preserves temporal causality and ensures that signature coefficients are computed on finite, well\-ordered trajectories without incorporating future information\.

Each modality is first embedded in a dedicated vector space, optionally compressed, and concatenated across time\. The resulting multimodal sequences are encoded using the signature transform, producing fixed\-dimensional representations that capture higher\-order temporal interactions\. Moreover, the signature transform is particularly well suited to handle irregularly sampled time points and heterogeneous data modalities, both of which constitute major challenges in real\-world machine learning settings\.

Finally, these signature\-derived features are used to fit a LASSO\-regularized Cox model, producing individualized risk scores while ensuring sparsity, strong predictive performance, and computational efficiency\.

An overview of the complete pipeline – from embedding extraction and temporal encoding to geometric clustering and survival estimation – is illustrated in Figure[1](https://arxiv.org/html/2608.16972#S2.F1)\.

![Refer to caption](https://arxiv.org/html/2608.16972v1/pipeline_multimodal.png)Figure 1:Overview of the MultiSigBERT pipeline\.Step 1: Multimodal representation\.Narrative clinical reports are converted into sentence embeddings and combined with structured variables observed at timest1,…,tNt\_\{1\},\\dots,t\_\{N\}to form multimodal vectorsvtv\_\{t\}\.Step 2: Landmark restriction\.Only observations within the window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]are retained to avoid temporal information leakage\.Step 3: Signature encoding\.The truncated sequence\(vt\)t\(v\_\{t\}\)\_\{t\}is encoded via the path signature transform, producing a fixed\-dimensional representation𝕊i\\mathbb\{S\}\_\{i\}capturing temporal interactions across modalities\.Step 4: Survival modeling\.A LASSO\-regularized Cox model is fitted on the signature features to estimate individualized risk scores\.Step 5: Evaluation\.Performance is assessed using the C\-index and the Brier score\.
### 2\.2Multimodal Embedding Extraction

Clinical datasets combine structured numerical variables with unstructured data such as free\-text reports\. To integrate these heterogeneous modalities into a unified survival framework, each input must first be transformed into a numerical vector representation\.

Let’s first consider narrative medical reports\. These longitudinal documents capture the patient’s clinical follow\-up, including consultation reports, hospitalization summaries, and medical observations recorded over time\. We encode them using OncoBERT\[[26](https://arxiv.org/html/2608.16972#bib.bib26)\], a CamemBERT\-based\[[19](https://arxiv.org/html/2608.16972#bib.bib19)\]language model fine\-tuned on oncology notes from the Léon Bérard Center\. This domain\-specific fine\-tuning improves the semantic representation of oncology terminology and clinical context\. Letℛi​\(t\)\\mathscr\{R\}\_\{i\}\(t\)denote the raw clinical report associated with patientiiat timett\. The OncoBERT encoderΦOncoBERT\\Phi\_\{\\mathrm\{OncoBERT\}\}maps this report into a sequence of contextualized token embeddings:

\(ew\)w∈ℛi​\(t\)=ΦOncoBERT​\(ℛi​\(t\)\),\\big\(e\_\{w\}\\big\)\_\{w\\in\\mathscr\{R\}\_\{i\}\(t\)\}=\\Phi\_\{\\mathrm\{OncoBERT\}\}\(\\mathscr\{R\}\_\{i\}\(t\)\),whereew∈ℝpe\_\{w\}\\in\\mathbb\{R\}^\{p\}denotes the contextualized embedding associated with tokenww, withp=768p=768\. A fixed\-dimensional sentence representationvi​\(t\)∈ℝpv\_\{i\}\(t\)\\in\\mathbb\{R\}^\{p\}is then obtained from these contextualized token embeddings using the Smooth Inverse Frequency \(SIF\) pooling strategy\[[2](https://arxiv.org/html/2608.16972#bib.bib2)\]\. Specifically,

vi​\(t\)=1\|ℛi​\(t\)\|​∑w∈ℛi​\(t\)af⁡\(w\)\+a​ew,v\_\{i\}\(t\)=\\frac\{1\}\{\|\\mathscr\{R\}\_\{i\}\(t\)\|\}\\sum\_\{w\\in\\mathscr\{R\}\_\{i\}\(t\)\}\\frac\{a\}\{\\mathrm\{f\}\(w\)\+a\}\\,e\_\{w\},wheref⁡\(w\)\\mathrm\{f\}\(w\)denotes the frequency of wordww,\|ℛi​\(t\)\|\|\\mathscr\{R\}\_\{i\}\(t\)\|denotes the number of tokens in the report, andaais a smoothing parameter, typically set toa=10−3a=10^\{\-3\}\.

The resulting vectorvi​\(t\)v\_\{i\}\(t\)constitutes the sentence embedding associated with patientiiat timettand serves as the elementary observation in the subsequent longitudinal trajectory\. We adopt Smooth Inverse Frequency \(SIF\) embeddings rather than the standard CLS token representation\. Empirically, SIF consistently outperformed CLS in our retrospective experiments, yielding higher C\-index values \(0\.74 vs\. 0\.70\) and lower Brier scores\. This gain can be attributed to the re\-weighting and averaging mechanism of SIF, which mitigates the influence of frequent or uninformative tokens and produces more stable sentence representations\. In addition, SIF naturally accommodates long clinical documents by aggregating token embeddings beyond the 512\-token limitation of BERT models\. For a detailed methodological and empirical discussion, we refer the reader to\[[21](https://arxiv.org/html/2608.16972#bib.bib21)\]\. A patient withNiN\_\{i\}reports is thus represented by the time\-indexed sequence of sentence embeddings

\(vi​\(t1\),…,vi​\(tNi\)\),vi​\(tj\)∈ℝp,\\big\(v\_\{i\}\(t\_\{1\}\),\\dots,v\_\{i\}\(t\_\{N\_\{i\}\}\)\\big\),\\qquad v\_\{i\}\(t\_\{j\}\)\\in\\mathbb\{R\}^\{p\},which can equivalently be viewed as an element ofℝNi×p\\mathbb\{R\}^\{N\_\{i\}\\times p\}\.

Since the signature transform scales exponentially with the input dimension, dimensionality reduction is required\. We apply Principal Component Analysis \(PCA\) on the training embeddings and retain the firstrrprincipal components, yielding a projection matrixRcomp∈ℝr×p\.R\_\{\\mathrm\{comp\}\}\\in\\mathbb\{R\}^\{r\\times p\}\.Each sentence embedding is then compressed according to

v~i​\(t\)=Rcomp⋅vi​\(t\)∈ℝr\.\\widetilde\{v\}\_\{i\}\(t\)=R\_\{\\mathrm\{comp\}\}\\cdot v\_\{i\}\(t\)\\in\\mathbb\{R\}^\{r\}\.The resulting trajectory\(v~i​\(t1\),…,v~i​\(tNi\)\)\\big\(\\widetilde\{v\}\_\{i\}\(t\_\{1\}\),\\dots,\\widetilde\{v\}\_\{i\}\(t\_\{N\_\{i\}\}\)\\big\)is subsequently used for temporal encoding through the signature transform\. The dimensionrris selected empirically to balance predictive performance and computational cost\. In our experiments, performance stabilized fromr=25r=25, which we then adopt in the sequel\.

### 2\.3Integration of Structured Clinical Variables

Beyond text, patients are characterized by longitudinal structured variables beingWeight,Karnofsky Index,Blood Pressure, andPulse\(meaning that the number of structured clinical covariates is given byK=4K=4\)\. These measurements form irregular time series that reflect physiological evolution\. To integrate them with textual embeddings, we construct synchronized multimodal trajectories\. For each structured variablexkx^\{k\},k=1,…,Kk=1,\\dots,K, observations are first sorted chronologically; missing internal values are then linearly interpolated, boundary values are propagated forward or backward when necessary, and all variables are finally standardized to zero mean and unit variance\. At each timestamptt, the compressed textual representation of patientiiis given byv~i​\(t\)∈ℝr\\widetilde\{v\}\_\{i\}\(t\)\\in\\mathbb\{R\}^\{r\}, while the structured measurements are represented by

xi​\(t\)=\(xi1​\(t\),…,xiK​\(t\)\)∈ℝK\.x\_\{i\}\(t\)=\\Big\(x\_\{i\}^\{1\}\(t\),\\dots,x\_\{i\}^\{K\}\(t\)\\Big\)\\in\\mathbb\{R\}^\{K\}\.In our setting, a patientiiwill be represented by the synchronized multimodal trajectory

Xi:\[0,Ti\]⟶ℝq,X\_\{i\}:\[0,T\_\{i\}\]\\longrightarrow\\mathbb\{R\}^\{q\},whose coordinates define a collection ofqqone\-dimensional paths evolving over time\. The ambient dimensionqqis obtained by concatenating the compressed embedding representation of sizerr, theKKstructured clinical variables, and the temporal coordinate required to preserve the chronological ordering of observations, yieldingq=r\+K\+1\.q=r\+K\+1\.The multimodal observation at timettis defined by concatenation:

Xi​\(t\)=\(t,v~i​\(t\),xi1​\(t\),…,xiK​\(t\)\)∈ℝq,X\_\{i\}\(t\)=\\Big\(t,\\widetilde\{v\}\_\{i\}\(t\),x\_\{i\}^\{1\}\(t\),\\dots,x\_\{i\}^\{K\}\(t\)\\Big\)\\in\\mathbb\{R\}^\{q\},where the first coordinate corresponds to the temporal component,v~i​\(t\)∈ℝr\\widetilde\{v\}\_\{i\}\(t\)\\in\\mathbb\{R\}^\{r\}denotes the compressed textual representation extracted from the clinical report, andxik​\(t\)x\_\{i\}^\{k\}\(t\)denotes the value of thekk\-th structured clinical variable at timett\. Given the observation times0≤t1<⋯<tNi≤Ti,0\\leq t\_\{1\}<\\cdots<t\_\{N\_\{i\}\}\\leq T\_\{i\},the trajectory of patientiican be represented equivalently as the matrix

Xi=\(Xi​\(t1\),…,Xi​\(tNi\)\)⊤∈ℝNi×q,X\_\{i\}=\\big\(X\_\{i\}\(t\_\{1\}\),\\dots,X\_\{i\}\(t\_\{N\_\{i\}\}\)\\big\)^\{\\top\}\\in\\mathbb\{R\}^\{N\_\{i\}\\times q\},\(1\)whose rows correspond to successive multimodal observations and whose columns correspond to the different coordinates of the trajectory\. This representation jointly encodes the semantic evolution extracted from clinical narratives and the temporal evolution of structured physiological measurements over theNiN\_\{i\}available observation times\.

In the next section, the signature transform is applied to the trajectoryXiX\_\{i\}in order to extract higher\-order temporal features describing the patient’s longitudinal evolution\.

### 2\.4Signature Feature Extraction

The signature of a path\[[5](https://arxiv.org/html/2608.16972#bib.bib5)\], later adapted to rough path theory by Lyons\[[18](https://arxiv.org/html/2608.16972#bib.bib18)\], provides a systematic way of encoding sequential data through collections of iterated integrals\. To ensure uniqueness of the signature representation, a monotone coordinate is typically appended to the path, most commonly the time component\. WritingXi​\(t\)=\(Xi1​\(t\),…,Xiq​\(t\)\),X\_\{i\}\(t\)=\\bigl\(X\_\{i\}^\{1\}\(t\),\\dots,X\_\{i\}^\{q\}\(t\)\\bigr\),the iterated\-integral signature coordinate associated with the multi\-index\(j1,…,jℓ\)∈\{1,…,q\}ℓ\(j\_\{1\},\\dots,j\_\{\\ell\}\)\\in\\\{1,\\dots,q\\\}^\{\\ell\}is defined by

S\(Xi\)0,t\(j1,…,jℓ\)=∫0<t1<⋯<tℓ<tdXij1\(t1\)⋯dXijℓ\(tℓ\)\.S\(X\_\{i\}\)\_\{0,t\}^\{\(j\_\{1\},\\dots,j\_\{\\ell\}\)\}=\\int\_\{0<t\_\{1\}<\\cdots<t\_\{\\ell\}<t\}\\mathrm\{d\}X\_\{i\}^\{j\_\{1\}\}\(t\_\{1\}\)\\cdots\\mathrm\{d\}X\_\{i\}^\{j\_\{\\ell\}\}\(t\_\{\\ell\}\)\.
The collection of these features is organized in tensor form which uniquely encodes the path and is defined as:

Sℓ​\(v\)=\(S​\(v\)0,T\(j1,…,jℓ\)\)\(j1,…,jℓ\)∈\{1,…,q\}ℓ∈\(ℝq\)⊗ℓ\.S^\{\\ell\}\(v\)=\\Big\(S\(v\)\_\{0,T\}^\{\(j\_\{1\},\\ldots,j\_\{\\ell\}\)\}\\Big\)\_\{\(j\_\{1\},\\ldots,j\_\{\\ell\}\)\\in\\\{1,\\dots,q\\\}^\{\\ell\}\}\\in\(\\mathbb\{R\}^\{q\}\)^\{\\otimes\\ell\}\.Thus, the truncated signature up to orderLLnaturally belongs to the truncated tensor algebra𝒯≤L​\(ℝq\)=⨁ℓ=0L\(ℝq\)⊗ℓ\\mathcal\{T\}^\{\\leq L\}\(\\mathbb\{R\}^\{q\}\)=\\bigoplus\\limits\_\{\\ell=0\}^\{L\}\\,\(\\mathbb\{R\}^\{q\}\)^\{\\otimes\\ell\}of orderLLoverℝq\\mathbb\{R\}^\{q\}:

S≤L​\(v\):=\(Sℓ​\(v\)\)ℓ=0L∈𝒯≤L​\(ℝp\)\.S^\{\\leq L\}\(v\):=\\big\(S^\{\\ell\}\(v\)\\big\)\_\{\\ell=0\}^\{L\}\\in\\mathcal\{T\}^\{\\leq L\}\(\\mathbb\{R\}^\{p\}\)\.
In the remainder of this work, we explicitly denote the truncated\-signature operator of orderLLby

Sig\(L\)⁡\(Xi\):=S≤L​\(Xi\),\\operatorname\{Sig\}^\{\(L\)\}\(X\_\{i\}\):=S^\{\\leq L\}\(X\_\{i\}\),denoting the collection of all iterated\-integral coordinates of the multimodal trajectoryXiX\_\{i\}up to levelLL\.

In addition to encoding temporal dynamics, this approach handles sequences of varying lengths and is invariant to translation and temporal reparameterization \(see\[[6](https://arxiv.org/html/2608.16972#bib.bib6)\]\), making it well\-suited for patients with different study entry points and durations\. Finally, for a given patientii, the collection of signature covariates extracted from the multimodal trajectoryXiX\_\{i\}, truncated at orderm=2m=2, is denoted by𝕊i=Sig\(2\)⁡\(Xi\)\\mathbb\{S\}\_\{i\}=\\operatorname\{Sig\}^\{\(2\)\}\(X\_\{i\}\)\. More explicitly, the truncated signature can be written as

𝕊i=\(1,S\(1\),…,S\(q\),S\(1,1\),…,S\(q,q\)\)i\.\\mathbb\{S\}\_\{i\}=\\Big\(1,\\,S^\{\(1\)\},\\ldots,S^\{\(q\)\},\\,S^\{\(1,1\)\},\\ldots,S^\{\(q,q\)\}\\Big\)\_\{i\}\.Each patient trajectory, initially represented as a time series, is therefore summarized by a fixed\-dimensional vector of signature coefficients𝕊i\\mathbb\{S\}\_\{i\}encoding its temporal dynamics\. These signature covariates provide a structured representation of longitudinal evolution in a Euclidean space\. This representation avoids the direct manipulation of irregular time series and enables the use of standard machine learning models for downstream tasks while preserving temporal consistency\.

### 2\.5Survival Analysis Modeling

The Cox proportional hazards model\[[7](https://arxiv.org/html/2608.16972#bib.bib7)\]remains the gold standard in survival analysis, notably because it naturally accounts for right\-censored observations, i\.e\., patients for whom the event of interestTT\(such as death or relapse\) has not yet occurred during the observation period\. In this work, we therefore adopt the Cox model to illustrate the impact of combining textual data with clinical variables\. All censored patients still contribute to the likelihood estimation, helping to reduce bias and improve the robustness of predictions\. The goal is to estimate the probability of a patient surviving beyond timett, noted as𝒮⁡\(t∣𝕊\):=ℙ⁡\(T≥t∣𝕊\)\\mathcal\{S\}\(t\\mid\\mathbb\{S\}\):=\\mathbb\{P\}\\big\(T\\geq t\\mid\\mathbb\{S\}\\big\)when knowing their covariates𝕊\\mathbb\{S\}\. This estimation relies on the key concept of instantaneous hazard ratehh, which quantifies the infinitesimal probability of the event occurring atttand is related to survival through the following equation:

𝒮\(t∣𝕊\)=exp\(−∫th\(s∣𝕊\)ds\)\.\\mathcal\{S\}\(t\\mid\\mathbb\{S\}\)=\\exp\\left\(\-\\int^\{t\}h\(s\\mid\\mathbb\{S\}\)\\,\\mathrm\{d\}s\\right\)\.David R\. Cox proposed the generalized linear model:

h⁡\(t∣𝕊\)=h0​\(t\)⋅exp⁡\(𝕊⋅𝜷\),h\(t\\mid\\mathbb\{S\}\)\\,=\\,h\_\{0\}\(t\)\\cdot\\exp\\big\(\\mathbb\{S\}\\cdot\\boldsymbol\{\\beta\}\\big\),where𝜷∈ℝQ\\boldsymbol\{\\beta\}\\in\\mathbb\{R\}^\{Q\}is the vector of parameters to be estimated, andh0h\_\{0\}is the baseline hazard, common to all patients, as estimated by\[[4](https://arxiv.org/html/2608.16972#bib.bib4)\]\. We defineη:=𝕊⋅𝜷\\eta:=\\mathbb\{S\}\\cdot\\boldsymbol\{\\beta\}, referred to as the risk score\. Estimating𝜷\\boldsymbol\{\\beta\}involves managing a substantial number of covariates\. As mentioned earlier, this is due to the signature transform, which generates a high\-dimensional feature space: forqqinput channels and a truncation levelLL, the number of resulting signature coefficients scales asO⁡\(qL\)O\(q^\{L\}\)\. Even after dimensionality reduction, the resulting covariate space remains large\. To reduce the risk of overfitting and improve model stability, we apply the LASSO \(Least Absolute Shrinkage and Selection Operator\) regularization to the Cox model, as originally introduced by Robert Tibshirani in\[[24](https://arxiv.org/html/2608.16972#bib.bib24)\]:

𝜷^∈argmax𝜷​log⁡PL​\(𝜷\)−λ​‖𝜷‖1,\\widehat\{\\boldsymbol\{\\beta\}\}\\in\\underset\{\\boldsymbol\{\\beta\}\}\{\\operatorname\{argmax\}\}\\ \\log\\text\{PL\}\(\\boldsymbol\{\\beta\}\)\-\\lambda\\\|\\boldsymbol\{\\beta\}\\\|\_\{1\},\(2\)wherePL​\(𝜷\)\\text\{PL\}\(\\boldsymbol\{\\beta\}\)is the partial likelihood defined and fully detailed in\[[8](https://arxiv.org/html/2608.16972#bib.bib8)\], andλ\>0\\lambda\>0denotes the regularization parameter\.‖𝜷‖1\\\|\\boldsymbol\{\\beta\}\\\|\_\{1\}is theℓ1\\ell\_\{1\}\-norm of the parameters𝜷\\boldsymbol\{\\beta\}\. The impact of LASSO regularization is twofold: it shrinks some coefficients towards zero, effectively removing less relevant covariates, and it selects only the most important predictors for survival, enhancing model stability\.

Beyond its statistical advantages, this sparsity pattern provides structural insight: only a limited subset of signature coordinates appears to carry prognostic information\. In other words, although the signature transform embeds patient trajectories into a high\-dimensional feature space, the effective predictive signal lies along a small number of directions\. This suggests that progression patterns may have a low intrinsic dimensionality within the signature representation, highlighting the importance of both regularization and structured feature extraction in our framework\. Moreover, by enforcing sparsity, the LASSO\-regularized Cox model significantly reduces the number of active covariates, leading to faster computational performance\. This suggests that the model achieves a favorable balance between overfitting and underfitting, leveraging a compact and efficient representation of the risk factors while maintaining strong predictive power\.

To put this idea into practice, model parameters are obtained by maximizing the penalized objective function in \([2](https://arxiv.org/html/2608.16972#S2.E2)\), whose explicit, form whenlog\\logapplied, is given by:

ℒ\(𝜷\)=∑i:δi=1\[𝕊i𝜷−log∑j∈ℛiexp\(𝕊j𝜷\)\]−λ∑k=1Q\|βk\|\.\\mathcal\{L\}\(\\boldsymbol\{\\beta\}\)=\\sum\_\{i:\\delta\_\{i\}=1\}\\left\[\\mathbb\{S\}\_\{i\}\\boldsymbol\{\\beta\}\-\\log\\sum\_\{j\\in\\mathcal\{R\}\_\{i\}\}\\exp\\Big\(\\mathbb\{S\}\_\{j\}\\boldsymbol\{\\beta\}\\Big\)\\right\]\-\\lambda\\sum\_\{k=1\}^\{Q\}\|\\beta\_\{k\}\|\.Here,δi≡𝟙\{Di=1\}∈\{0,1\}\\delta\_\{i\}\\equiv\\mathbbm\{1\}\_\{\\\{D\_\{i\}=1\\\}\}\\in\\\{0,1\\\}indicates whether the event \(e\.g\., death\) has been observed for patientii, with associated study durationTiT\_\{i\}\. The setℛi\\mathcal\{R\}\_\{i\}denotes the risk set, i\.e\., the individuals still at risk at timeTiT\_\{i\}, formally defined asℛi=\{j:Tj≥Ti\}\\mathcal\{R\}\_\{i\}=\\\{j:T\_\{j\}\\geq T\_\{i\}\\\}\.

Finally, the estimated risk score under LASSO regularization is obtained simply as the dot product

η^=𝕊⋅𝜷^\.\\widehat\{\\eta\}=\\mathbb\{S\}\\cdot\\widehat\{\\boldsymbol\{\\beta\}\}\.Each patient characterized by a sequence of clinical reports and associated sequential covariates is assigned an individualized risk scoreη^\\widehat\{\\eta\}that summarizes the temporal evolution of their clinical trajectory\. Overall, this pipeline provides a straightforward framework to integrate heterogeneous EHR modalities into a unified representation suitable for survival analysis\.

To ensure a well\-defined prediction task and avoid temporal information leakage, it is however necessary to construct these representations using only information available up to a given prediction time\. In the next section, we therefore introduce a landmark design that restricts the computation of signature coefficients to observations occurring before a fixed time point, ensuring that the model is trained exclusively on past information\.

### 2\.6Landmark Modeling for Dynamic Survival Prediction

Landmarking provides a principled framework for dynamic survival prediction by conditioning on a fixed prediction time𝐋\\boldsymbol\{\\mathrm\{L\}\}\[[1](https://arxiv.org/html/2608.16972#bib.bib1),[25](https://arxiv.org/html/2608.16972#bib.bib25)\]\. Rather than using the entire longitudinal process, including information recorded after the prediction time, the analysis is restricted to the landmark risk set

ℛ𝐋=\{i:Ei≥𝐋\},\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}=\\bigl\\\{i:E\_\{i\}\\geq\\boldsymbol\{\\mathrm\{L\}\}\\bigr\\\},\(3\)whereEi=min⁡\(Ti,Ci\)E\_\{i\}=\\min\(T\_\{i\},C\_\{i\}\)denotes the observed survival time, obtained from the true event timeTiT\_\{i\}and censoring timeCiC\_\{i\}\. Consequently, only patients who remain under observation at time𝐋\\boldsymbol\{\\mathrm\{L\}\}contribute to the landmark analysis\. For each patienti∈ℛ𝐋i\\in\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}, survival is re\-indexed relative to the landmark:

Ri=Ei−𝐋≥0\.R\_\{i\}=E\_\{i\}\-\\boldsymbol\{\\mathrm\{L\}\}\\geq 0\.The corresponding landmark event indicator is defined byδi\(𝐋\)=δi1\{Ei\>𝐋\}\\delta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)=\\delta\_\{i\}\\,\\mathbbm\{1\}\_\{\\\{E\_\{i\}\>\\boldsymbol\{\\mathrm\{L\}\}\\\}\}whereδi=𝟙\{Ti≤Ci\}\\delta\_\{i\}=\\mathbbm\{1\}\_\{\\\{T\_\{i\}\\leq C\_\{i\}\\\}\}still denotes the usual event indicator\. This formulation ensures that both the outcome and the covariates are defined relative to the same prediction time𝐋\\boldsymbol\{\\mathrm\{L\}\}, thereby preventing temporal leakage and guaranteeing that predictions are conditional on survival up to the landmark\. An illustration of the landmark principle is provided in Figure[2](https://arxiv.org/html/2608.16972#S2.F2)\.

TimeBaselineWindow\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]Ri=Ei−𝐋R\_\{i\}=E\_\{i\}\-\\boldsymbol\{\\mathrm\{L\}\}𝐋−𝐰\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\}𝐋\\boldsymbol\{\\mathrm\{L\}\}𝐋\+τ\\boldsymbol\{\\mathrm\{L\}\}\+\\tauFigure 2:Landmark design: features are constructed from the backward window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]and survival is re\-indexed byRi=Ei−𝐋R\_\{i\}=E\_\{i\}\-\\boldsymbol\{\\mathrm\{L\}\}\. The prediction task is to estimate the conditional survival probabilityℙ⁡\(Ti\>𝐋\+τ∣𝕏i​\(𝐋\)\)\\mathbb\{P\}\(T\_\{i\}\>\\boldsymbol\{\\mathrm\{L\}\}\+\\tau\\mid\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\), using only information observed before the landmark time\.Let’s apply this setup to our context\. For a fixed landmark time𝐋\\boldsymbol\{\\mathrm\{L\}\}, feature extraction is restricted to the backward window\[𝐋−𝐰,𝐋\],\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\],thereby ensuring that only information available prior to the prediction time is used\. Denoting byXi​\(𝐋\)X\_\{i\}\{\(\\boldsymbol\{\\mathrm\{L\}\}\)\}the restriction of the trajectoryXiX\_\{i\}\(defined in[1](https://arxiv.org/html/2608.16972#S2.E1)\) to this interval, the landmark\-specific signature representation is defined as

𝕊i​\(𝐋\)=Sig\(2\)⁡\(Xi​\(𝐋\)\)\.\\mathbb\{S\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)=\\operatorname\{Sig\}^\{\(2\)\}\\\!\\left\(X\_\{i\}\{\(\\boldsymbol\{\\mathrm\{L\}\}\)\}\\right\)\.
To account for patients with limited clinical history prior to the landmark, we introduce the binary indicator

ζi\(𝐋\)=𝟙\{𝐋−tidiag<𝐰\},\\zeta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)=\\mathbbm\{1\}\_\{\\left\\\{\\boldsymbol\{\\mathrm\{L\}\}\-t\_\{i\}^\{\\mathrm\{diag\}\}<\\boldsymbol\{\\mathrm\{w\}\}\\right\\\}\},wheretidiagt\_\{i\}^\{\\mathrm\{diag\}\}denotes the diagnosis time \(or, equivalently, the first available clinical observation\)\. This indicator identifies patients whose available history before𝐋\\boldsymbol\{\\mathrm\{L\}\}is shorter than the landmark window\. This indicator is particularly relevant in landmark modeling, as some patients may have entered the study less than𝐰\\boldsymbol\{\\mathrm\{w\}\}time units before the landmark and therefore lack a complete history over the window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]\. The final landmark covariate vector is obtained by concatenation:

𝕏i​\(𝐋\)=\[𝕊i​\(𝐋\),ζi​\(𝐋\)\]⊤∈ℝQ\+1\.\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)=\[\\mathbb\{S\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\),\\ \\zeta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\]^\{\\top\}\\in\\mathbb\{R\}^\{Q\+1\}\.The resulting landmark dataset consists of triplets\(𝕏i​\(𝐋\),δi​\(𝐋\),Ri\),\\Bigl\(\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\),\\,\\delta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\),\\,R\_\{i\}\\Bigr\),for eachi∈ℛ𝐋i\\in\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}, which are used to estimate conditional survival beyond the landmark time\. More precisely, the prediction task consists of estimating

ℙ⁡\(Ti\>𝐋\+τ\|𝕏i​\(𝐋\)\),\\mathbb\{P\}\\Bigl\(T\_\{i\}\>\\boldsymbol\{\\mathrm\{L\}\}\+\\tau\\;\\Big\|\\;\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\\Bigr\),that is, the probability that patientiisurvives an additional horizonτ\\taubeyond the landmark time, given all information available up to𝐋\\boldsymbol\{\\mathrm\{L\}\}\.

This framework is particularly well suited to longitudinal clinical data and signature\-based representations\. By construction, it ensures temporal coherence, prevents information leakage, accommodates irregular follow\-up patterns, and allows patients with heterogeneous observation histories to be compared at a common prediction time\.

## 3Experiments

### 3\.1Cohort

This study complies with the General Data Protection Regulation \(GDPR\) and falls within the scope of scientific research conducted in the legitimate interest of cancer research, in accordance with Articles 6\.1\.f and 9\.2\.j of Regulation \(EU\) No\. 2016/679\. This project has been officially registered under the MR004 declaration \(V3\.2, 23/08/2021\) at the Léon Bérard Center, ensuring compliance with legal and ethical standards for processing health data\. The data have been carefully anonymized and can only be used within the framework of this study\. No patient were opposed to this study\. To ensure the reliability of our data, we selected a study cohort consisting of patients hospitalized – at least once – at the Léon Bérard Center from 2000 to 2024, with comprehensive follow\-up throughout their medical care to ensure data completeness and accuracy\.

All results reported in this section are obtained under a landmark design with landmark time𝐋=36\\boldsymbol\{\\mathrm\{L\}\}=36months and backward window size𝐰=12\\boldsymbol\{\\mathrm\{w\}\}=12months\. In other words, we consider patients who remain at risk three years after their initial diagnosis or first hospitalization, and construct covariates using information collected during the one\-year period preceding the landmark\.

The dataset consists of longitudinal medical reports collected from a comprehensive oncology cohort\. It includes 2,527 patients and a total of 124,049 clinical reports, with an average of 49 reports per patient, reflecting dense longitudinal follow\-up\. Among these patients, 1,629 experienced the event of interest and 898 were right\-censored\.

The cohort covers a wide spectrum of cancer types\. The most represented diagnoses are breast cancer \(782 patients, 31\.0%\), and to name a few more, gynecological cancers \(232, 9\.2%\), intestinal cancers \(227, 9\.0%\), prostate cancer \(201, 8\.0%\), and lung cancer \(96, 3\.8%\)\. Additional cases include other oncological subtypes, ensuring heterogeneity in disease trajectories and survival patterns\. The longitudinal structure of the data makes it particularly suitable for landmark modeling\. At each landmark time𝐋\\boldsymbol\{\\mathrm\{L\}\}, only patients still at risk are considered, and covariates are constructed from the backward window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\], ensuring temporal coherence and preventing information leakage\. Descriptive statistics and additional cohort characteristics \(cancer types, number of reports per patient, study duration, diagnosis dates, and Kaplan–Meier curves\) are available in the associated GitHub repository \([https://github\.com/MINCHELLA\-Paul/MultiSigBERT/tree/main/results/descriptive\_statistics](https://github.com/MINCHELLA-Paul/MultiSigBERT/tree/main/results/descriptive_statistics)\)\.

### 3\.2Hyperparameter Search

Our experimental setup is designed to ensure reproducibility, robustness, and realistic evaluation in a complex real\-world clinical context\. The NLP model used in this study OncoBERT is previously fine\-tuned on oncology\-specific clinical notes\. It is used as is throughout the experiments, without further task\-specific adaptation, in order to reflect realistic deployment scenarios\. The survival model was trained on a cohort of1,2641,264patients \(9,6969,696reports\) and evaluated on a separate test set of1,2631,263patients \(9,7529,752reports\), using a structured and stratified train\-test split to preserve temporal and distributional consistency\. To calibrate the model, we conducted a grid search for the LASSO regularization parameterλ\\lambdawithin the range\[0\.01,16\]\[0\.01,16\], using a fixed step size of0\.010\.01\. We selected the value that maximized the cross\-validation concordance index \(C\-index\) averaged over five independent validation folds, within the training dataset\.

At this step, the longitudinal data were transformed using the signature method, effectively eliminating any temporal constraints that could arise when subdividing the dataset for cross\-validation\. The selection criterion aimed to maximize the C\-index through five\-fold cross\-validation\. Specifically, for each candidate value ofλ\\lambda, the model was trained on a partition of the training cohort and evaluated on held\-out subsets – within the training set –, maintaining equal proportions across folds\.

The optimal value was then chosen as the one yielding the highest mean C\-index, promoting robust generalization and avoiding overfitting\. This tuning process was crucial for balancing model sparsity and predictive accuracy\.

Scenario𝝀\\boldsymbol\{\\lambda\}C\-index Val\.↑\\uparrowIBS@3y↓\\downarrowReports Only5\.155\.150\.747​\(sd​0\.027\)0\.747\\;\(\\mathrm\{sd\}\\;0\.027\)0\.1280​\(sd​0\.004\)0\.1280\\;\(\\mathrm\{sd\}\\;0\.004\)Reports & Structured Variables4\.604\.600\.743​\(sd​0\.025\)0\.743\\;\(\\mathrm\{sd\}\\;0\.025\)0\.1280​\(sd​0\.004\)0\.1280\\;\(\\mathrm\{sd\}\\;0\.004\)Table 1:Optimal regularization LASSO\-parameterλ\\lambdaselected by validation for each modeling scenario\. Performance is reported as the mean C\-index on the validation sets with its standard deviation \(higher is better\) and the Integrated Brier Score at 3 years \(IBS@3y, lower is better\)\.
### 3\.3Performance Metrics

Model validation is performed using repeated random splitting: the test set is divided into ten disjoint subsets, and evaluation is repeated independently on each split\. Reported values correspond to the mean and standard deviation across splits\.

We first assess discrimination using the concordance index \(C\-index\)\[[12](https://arxiv.org/html/2608.16972#bib.bib12)\], computed on the landmark cohortℛ𝐋\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}from \([3](https://arxiv.org/html/2608.16972#S2.E3)\)\. The C\-index measures the proportion of correctly ordered comparable pairs\. If patientjjexperiences the event before patientii, a well\-specified model should assign a higher risk scoreη^j\>η^i\\widehat\{\\eta\}\_\{j\}\>\\widehat\{\\eta\}\_\{i\}\. Using the landmark outcomes\(Ri,δi​\(𝐋\)\)\(R\_\{i\},\\delta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\), it can be written as

C​\-​index=∑i,j∈ℛ𝐋𝟙\{Rj<Ri\}𝟙\{η^j\>η^i\}δj\(𝐋\)∑i,j∈ℛ𝐋𝟙\{Rj<Ri\}δj\(𝐋\)\.\\mathrm\{C\}\\text\{\-\}\\mathrm\{index\}=\\frac\{\\sum\\limits\_\{i,j\\in\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}\}\\mathbbm\{1\}\_\{\\\{R\_\{j\}<R\_\{i\}\\\}\}\\mathbbm\{1\}\_\{\\\{\\widehat\{\\eta\}\_\{j\}\>\\widehat\{\\eta\}\_\{i\}\\\}\}\\delta\_\{j\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\}\{\\sum\\limits\_\{i,j\\in\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}\}\\mathbbm\{1\}\_\{\\\{R\_\{j\}<R\_\{i\}\\\}\}\\delta\_\{j\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\}\.A value of11indicates perfect ranking, whereas0\.50\.5corresponds to random performance\.

While discrimination remains important, calibration has become equally critical in modern survival modeling\. A model may rank patients correctly while providing inaccurate survival probabilities\. To assess calibration, we use the Brier Score \(BS\), which measures the squared difference between the predicted landmark survival probability𝒮^​\(t∣𝕏i​\(𝐋\)\)\\widehat\{\\mathcal\{S\}\}\(t\\mid\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\)and the observed landmark outcome\. To account for right censoring, we employ inverse probability of censoring weighting \(IPCW\)\[[11](https://arxiv.org/html/2608.16972#bib.bib11)\]:

BS\(t\)=1\|ℛ𝐋\|∑i∈ℛ𝐋\[𝟙\{Ri≤t,δi​\(𝐋\)=1\}𝒮^​\(t∣𝕏i​\(𝐋\)\)2G^​\(Ri\)\+𝟙\{Ri\>t\}\(1−𝒮^​\(t∣𝕏i​\(𝐋\)\)\)2G^​\(t\)\]\.\\mathrm\{BS\}\(t\)=\\frac\{1\}\{\|\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}\|\}\\sum\_\{i\\in\\mathcal\{R\}\_\{\\boldsymbol\{\\mathrm\{L\}\}\}\}\\Bigg\[\\mathbbm\{1\}\_\{\\\{R\_\{i\}\\leq t,\\ \\delta\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)=1\\\}\}\\frac\{\\widehat\{\\mathcal\{S\}\}\(t\\mid\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\)^\{2\}\}\{\\widehat\{G\}\(R\_\{i\}\)\}\+\\mathbbm\{1\}\_\{\\\{R\_\{i\}\>t\\\}\}\\frac\{\\Big\(1\-\\widehat\{\\mathcal\{S\}\}\(t\\mid\\mathbb\{X\}\_\{i\}\(\\boldsymbol\{\\mathrm\{L\}\}\)\)\\Big\)^\{2\}\}\{\\widehat\{G\}\(t\)\}\\Bigg\]\.\(4\)whereG^\\widehat\{G\}denotes the Kaplan–Meier estimator\[[14](https://arxiv.org/html/2608.16972#bib.bib14)\]of the censoring distribution\. In the landmark setting, the Brier Score evaluates the accuracy of predicted residual survival probabilities beyond the landmark time𝐋\\boldsymbol\{\\mathrm\{L\}\}\.

The Brier Score directly quantifies the accuracy of predicted survival probabilities\. This property is crucial in clinical applications, where predicted survival probabilities may influence treatment decisions, patient stratification, and follow\-up strategies\. A value substantially below the naive reference value of0\.250\.25– corresponding to a constant prediction of0\.50\.5for all individuals – indicates good calibration\. Finally, to summarize predictive accuracy over a given time interval rather than at a single time point, we use the Integrated Brier Score \(IBS\) that provides a global calibration measure by integrating \([4](https://arxiv.org/html/2608.16972#S3.E4)\) over a time interval\[τ1,τ2\]\[\\tau\_\{1\},\\tau\_\{2\}\]\. In practice, it can be evaluated up to a fixed horizon, for example 3 years, by settingτ1=0\\tau\_\{1\}=0andτ2=3​y\\tau\_\{2\}=3\\text\{y\}\.

### 3\.4Experimental Results

All results obtained under the landmark design \(𝐋=36\\boldsymbol\{\\mathrm\{L\}\}=36,𝐰=12\\boldsymbol\{\\mathrm\{w\}\}=12\) are summarized in Table[2](https://arxiv.org/html/2608.16972#S3.T2)\. The models adapted to this architecture include DeepSurv\[[15](https://arxiv.org/html/2608.16972#bib.bib15)\], CoxTime\[[16](https://arxiv.org/html/2608.16972#bib.bib16)\], and the time\-varying Cox model implemented via CoxTimeVaryingFitter\[[9](https://arxiv.org/html/2608.16972#bib.bib9)\], denoted CTVF\. CoxTime and CTVF are trained and applied directly on the longitudinal time\-series representation via Equation \([2\.3](https://arxiv.org/html/2608.16972#S2.Ex8)\), whereas DeepSurv operates on the fixed\-length feature representation obtained after computing the signature coefficients\. We consider two experimental configurations: \(i\)*Reports Only*, relying exclusively on signature features extracted from textual embeddings \(therefore, corresponding to the initial model SigBERT\[[21](https://arxiv.org/html/2608.16972#bib.bib21)\]\); and \(ii\)*Reports \+ Sequential Structured Data*, where interpolated longitudinal clinical variables are concatenated with textual embeddings prior to the signature transformation\.

Overall, as shown in Table[2](https://arxiv.org/html/2608.16972#S3.T2), MultiSigBERT achieves strong predictive performance under the landmark design\. In the*Reports Only*configuration, it reaches a mean test C\-index of0\.741\(sd 0\.027\), substantially outperforming several classical survival baselines such as CoxTime \(0\.646\) and CTVF \(0\.619\)\. It also clearly improves upon neural survival models based on aggregated representations, such as DeepSurv, which attains a mean C\-index of 0\.600\. In terms of calibration, MultiSigBERT yields a relatively low prediction error with anIBS@​3​y\\mathrm\{IBS\}\_\{@3y\}of0\.128\(sd 0\.006\), indicating accurate survival probability estimates across the evaluation horizon\.

ModelC\-index↑\\uparrow\(Test\)IBS↓\\downarrow\(Test\)MultiSigBERT \(RO\)0\.741 \[0\.724, 0\.758\]0\.1282 \[0\.1243, 0\.1322\]DeepSurv \(RO\)0\.600 \[0\.583, 0\.614\]0\.1512 \[0\.1406, 0\.1619\]CoxTime \(RO\)0\.646 \[0\.638, 0\.654\]0\.091 \[0\.081, 0\.101\]CTVF \(RO\)0\.619 \[0\.594, 0\.645\]0\.207 \[0\.185, 0\.229\]MultiSigBERT \(RS\)0\.743 \[0\.725, 0\.761\]0\.1339 \[0\.1293, 0\.1385\]DeepSurv \(RS\)0\.650 \[0\.620, 0\.676\]0\.1490 \[0\.1386, 0\.1593\]CoxTime \(RS\)0\.651 \[0\.640, 0\.661\]0\.091 \[0\.081, 0\.101\]CTVF \(RS\)0\.618 \[0\.594, 0\.645\]0\.209 \[0\.185, 0\.229\]Table 2:Test\-set performance under the landmark design \(Train = 1,263 patients; Test = 1,264 patients\)\. Values are reported as mean with 95% Jackknife confidence intervals in brackets\. RO: Reports Only; RS: Reports \+ Sequential Structured Data\.↑\\uparrowindicates that higher values are better;↓\\downarrowindicates that lower values are better\.When sequential structured covariates are incorporated, MultiSigBERT maintains essentially the same level of predictive performance\. The mean test C\-index slightly increases to0\.743\(sd 0\.029\), while the calibration error becomesIBS@​3​y=0\.134\\mathrm\{IBS\}\_\{@3y\}=0\.134\(sd 0\.007\)\. These results suggest that adding structured variables provides only marginal gains in discrimination while slightly degrading calibration\.

Compared with neural survival baselines such as DeepSurv, MultiSigBERT offers a substantially better overall trade\-off between discrimination and calibration\. Even when structured variables are included, DeepSurv reaches a mean C\-index of only 0\.650, remaining clearly below the performance achieved by MultiSigBERT\. Although CoxTime achieves the lowestIBS@​3​y\\mathrm\{IBS\}\_\{@3y\}\(around 0\.091\), its discrimination remains substantially lower \(C\-index≈0\.65\\approx 0\.65\), which limits its usefulness for clinical risk stratification where reliable patient ranking is essential\.

From a clinical perspective, this trade\-off is particularly important\. Effective decision support requires models that can both correctly rank patients by risk and provide reasonably calibrated survival probabilities\. In this respect, MultiSigBERT provides a robust compromise: it achieves substantially higher discrimination than traditional Cox\-based approaches while maintaining stable and interpretable survival predictions\.

Finally, the limited improvement observed when incorporating sequential structured variables suggests that a large part of the prognostic signal carried by routinely collected structured measurements is already implicitly captured in longitudinal clinical narratives\. Since these variables must be interpolated over the backward window\[𝐋−𝐰,𝐋\]\[\\boldsymbol\{\\mathrm\{L\}\}\-\\boldsymbol\{\\mathrm\{w\}\},\\boldsymbol\{\\mathrm\{L\}\}\]due to their irregular recording times, this interpolation introduces smoothing assumptions and may propagate measurement noise\. Consequently, the multimodal configuration provides only limited additional predictive value in this setting\.

## 4Conclusion

We introduced MultiSigBERT, a multimodal survival modeling framework that integrates longitudinal clinical narratives and structured variables within a unified temporal representation\. The proposed pipeline combines domain\-specific language embeddings concatenated with sequential structured variables, dimensionality reduction, path signature encoding, and sparse Cox regression, yielding a computationally efficient and interpretable approach for survival prediction\.

Experiments on a large real\-world oncology cohort show that MultiSigBERT achieves strong predictive performance under a landmark design, with a C\-index around 0\.743 \(sd 0\.029\) and stable calibration\. Importantly, the landmark framework ensures a well\-defined dynamic prediction task and prevents temporal information leakage by restricting feature construction to observations available prior to the prediction time, therefore providing a robust and reproducible pipeline for longitudinal survival modeling\.

The proposed architecture naturally supports multimodal integration\. By embedding heterogeneous clinical data into synchronized trajectories prior to signature encoding, the model captures temporal interactions across modalities in a principled geometric representation\. In addition, the aggregation strategy enables the use of an arbitrary number of clinical reports per patient while preserving temporal ordering\. The entire pipeline remains computationally lightweight and can be trained on standard CPU hardware, offering a favorable trade\-off between predictive performance, interpretability, and computational cost\.

Some limitations should nevertheless be acknowledged\. The structured variables considered in this study remain limited, and their irregular sampling requires interpolation within the landmark window\. Future work will investigate additional static covariates, such as tumor type, sex, or age at diagnosis, which can be straightforwardly incorporated into the proposed framework\.

Overall, MultiSigBERT provides a robust and scalable framework for multimodal survival analysis from longitudinal clinical data\. A key contribution of this work lies in the use of path signatures as a unified representation of heterogeneous temporal modalities\. Signature transforms encode temporal dependencies and higher\-order interactions into a fixed\-dimensional vector while naturally handling irregularly sampled trajectories, making them particularly suitable for real\-world clinical data\.

Our results also suggest that classical sparse survival models remain highly competitive when combined with expressive temporal representations\. The combination of landmarking, signature\-based feature extraction, and Cox–LASSO regularization achieved a favorable trade\-off between predictive performance, interpretability, computational efficiency, and robustness\. Future work will explore richer multimodal representations and external validation on independent oncology cohorts\.

#### Disclosure of Interests\.

The authors declare no potential conflicts of interest\.

## References

- \[1\]Anderson, J\., Cain, K\., Gelber, R\.: Analysis of survival by tumor response\. Journal of Clinical Oncology1\(11\), 710–719 \(1983\)\. https://doi\.org/10\.1200/JCO\.1983\.1\.11\.710
- \[2\]Arora, S\., Liang, Y\., Ma, T\.: A simple but tough\-to\-beat baseline for sentence embeddings\. In: International Conference on Learning Representations \(ICLR\) \(2017\), published as a conference paper at ICLR 2017
- \[3\]Bleistein, L\., Nguyen, V\.T\., Fermanian, A\., Guilloux, A\.: Dynamical survival analysis with controlled latent states \(2024\)\. arXiv:2401\.17077 \[stat\.ML\]\. https://arxiv\.org/abs/2401\.17077
- \[4\]Breslow, N\.E\.: Contribution to the discussion of the paper by D\. R\. Cox\. Journal of the Royal Statistical Society: Series B \(Methodological\)34\(2\), 216–217 \(1972\)
- \[5\]Chen, K\.T\.: Iterated integrals and exponential homomorphisms\. Proceedings of the London Mathematical Society3\(4\), 502–512 \(1954\)
- \[6\]Chevyrev, I\., Kormilitzin, A\.: A primer on the signature method in machine learning \(2025\)\. arXiv:1603\.03788 \[stat\.ML\]\. https://arxiv\.org/abs/1603\.03788
- \[7\]Cox, D\.R\.: Regression models and life\-tables\. Journal of the Royal Statistical Society: Series B \(Methodological\)34\(2\), 187–202 \(1972\)\. https://doi\.org/10\.1111/j\.2517\-6161\.1972\.tb00899\.x
- \[8\]Cox, D\.R\.: Partial likelihood\. Biometrika62\(2\), 269–276 \(1975\)\. https://doi\.org/10\.1093/biomet/62\.2\.269
- \[9\]Davidson\-Pilon, C\.: lifelines: Survival analysis in Python\. Journal of Open Source Software4\(40\), 1317 \(2019\)\. https://doi\.org/10\.21105/joss\.01317
- \[10\]Devaux, A\., Genuer, R\., Peres, K\., Proust\-Lima, C\.: Individual dynamic prediction of clinical endpoint from large dimensional longitudinal biomarker history: a landmark approach\.*BMC Medical Research Methodology*22\(1\), 188 \(2022\)\.[https://doi\.org/10\.1186/s12874\-022\-01660\-3](https://doi.org/10.1186/s12874-022-01660-3)
- \[11\]Graf, E\., Schmoor, C\., Sauerbrei, W\., Schumacher, M\.: Assessment and comparison of prognostic classification schemes for survival data\. Statistics in Medicine18\(17–18\), 2529–2545 \(1999\)
- \[12\]Harrell, F\.E\., Califf, R\.M\., Pryor, D\.B\., Lee, K\.L\., Rosati, R\.A\.: Evaluating the yield of medical tests\. Journal of the American Medical Association247\(18\), 2543–2546 \(1982\)
- \[13\]Jee, J\., Fong, C\., Pichotta, K\., et al\.: Automated real\-world data integration improves cancer outcome prediction\.*Nature*636, 728–736 \(2024\)\.[https://doi\.org/10\.1038/s41586\-024\-08167\-5](https://doi.org/10.1038/s41586-024-08167-5)
- \[14\]Kaplan, E\.L\., Meier, P\.: Nonparametric estimation from incomplete observations\. Journal of the American Statistical Association53\(282\), 457–481 \(1958\)\. https://doi\.org/10\.2307/2281868
- \[15\]Katzman, J\.L\., Shaham, U\., Cloninger, A\., Bates, J\., Jiang, T\., Kluger, Y\.: DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network\. BMC Medical Research Methodology18\(1\) \(2018\)\. https://doi\.org/10\.1186/s12874\-018\-0482\-1
- \[16\]Kvamme, H\., Borgan, Ø\., Scheel, I\.: Time\-to\-event prediction with neural networks and Cox regression\. Journal of Machine Learning Research20\(116\), 1–30 \(2019\)\. http://jmlr\.org/papers/v20/18\-424\.html
- \[17\]Lee, C\., Yoon, J\., van der Schaar, M\.: Dynamic\-DeepHit: A deep learning approach for dynamic survival analysis with competing risks based on longitudinal data\. IEEE Transactions on Biomedical Engineering \(2019\)\. https://par\.nsf\.gov/servlets/purl/10099761
- \[18\]Lyons, T\.J\.: Differential equations driven by rough signals\. Revista Matemática Iberoamericana14\(2\), 215–310 \(1998\)
- \[19\]Martin, L\., Muller, B\., Ortiz Suárez, P\.J\., Dupont, Y\., Romary, L\., de la Clergerie, É\., Seddah, D\., Sagot, B\.: CamemBERT: a tasty French language model\. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics\. Association for Computational Linguistics \(2020\)\. https://doi\.org/10\.18653/v1/2020\.acl\-main\.645
- \[20\]Mesinovic, M\., Watkinson, P\., Zhu, T\.: DySurv: dynamic deep learning model for survival analysis with conditional variational inference \(2024\)\. arXiv:2310\.18681 \[cs\.LG\]\. https://arxiv\.org/abs/2310\.18681
- \[21\]Minchella, P\., Verlingue, L\., Chrétien, S\., Vaucher, R\., Metzler, G\.: Sigbert: Combining narrative medical reports and rough path signature theory for survival prediction in oncology\. In: Machine Learning and Knowledge Discovery in Databases\. Applied Data Science Track\. ECML PKDD 2025\. Lecture Notes in Computer Science, vol\. part IX\. Springer \(2025\)
- \[22\]Pourjafari, E\., Ziaei, N\., Rezaei, M\.R\., Sameizadeh, A\., Shafiee, M\., Alavinia, M\., Abolghasemian, M\., Sajadi, N\.: Survival Seq2Seq: A survival model based on sequence\-to\-sequence architecture\.*arXiv preprint arXiv:2204\.04542*\(2022\)\.[https://arxiv\.org/abs/2204\.04542](https://arxiv.org/abs/2204.04542)
- \[23\]Signorelli, M\., Spitali, P\., Szigyarto, C\.A\., Tsonaka, R\.: Penalized regression calibration: A method for the prediction of survival outcomes using complex longitudinal and high\-dimensional data\.*Statistics in Medicine*40\(27\), 6178–6196 \(2021\)\.[https://doi\.org/10\.1002/sim\.9178](https://doi.org/10.1002/sim.9178)
- \[24\]Tibshirani, R\.: The Lasso method for variable selection in the Cox model\. Statistics in Medicine16\(4\), 385–395 \(1997\)\. https://doi\.org/10\.1002/\(SICI\)1097\-0258\(19970228\)16:4<385::AID\-SIM380\>3\.0\.CO;2\-3
- \[25\]van Houwelingen, H\.C\., Putter, H\.: Dynamic predicting by landmarking as an alternative for multi\-state modeling\. Lifetime Data Analysis14\(4\), 447–463 \(2008\)\. https://doi\.org/10\.1007/s10985\-008\-9099\-8
- \[26\]Vienne, R\., Filori, Q\., Susplugas, V\., Crochet, H\., Verlingue, L\.: Abstract 3475: Prediction of nausea or vomiting, and fatigue or malaise in cancer care\. Cancer Research84, 3475–3475 \(2024\)\. https://doi\.org/10\.1158/1538\-7445\.AM2024\-3475
- \[27\]Zhao, Y\., Hong, Q\., Zhang, X\., Deng, Y\., Wang, Y\., Petzold, L\.: BERTSurv: BERT\-based survival models for predicting outcomes of trauma patients\.*arXiv preprint arXiv:2103\.10928*\(2021\)\.[https://arxiv\.org/abs/2103\.10928](https://arxiv.org/abs/2103.10928)

## Supplementary

![Refer to caption](https://arxiv.org/html/2608.16972v1/Supplementary/figures/SUPP-repartition_cancer_eng.png)Figure 3:Distribution of cancer types in the study cohort\. The figure reports the number of patients for each cancer category\. Breast cancer represents the largest group \(782 patients\), followed by gynecological \(232\), intestinal \(227\), and prostate cancers \(201\)\. The remaining categories correspond to less frequent tumor types, reflecting the heterogeneous composition of the oncology cohort\.![Refer to caption](https://arxiv.org/html/2608.16972v1/Supplementary/figures/SUPP-source_repartition_eng.png)Figure 4:Distribution of report types in the clinical corpus\. The majority of documents correspond to consultation reports \(83,898\), followed by hospital stay reports \(33,966\)\. Additional categories include follow\-up consultations during treatment \(3,778\), consultation reports with procedures \(1,618\), and discharge reports \(789\)\. This distribution reflects the predominance of outpatient consultations in routine oncology follow\-up, while hospitalizations represent a smaller but clinically important subset of the longitudinal documentation\.![Refer to caption](https://arxiv.org/html/2608.16972v1/Supplementary/figures/SUPP-structured-data-hist.png)Figure 5:Empirical distributions of the sequential structured clinical variables used in the study: peripheral oxygen saturation \(PO\), systolic blood pressure \(TA\), Karnofsky performance index \(KAR\), and pulse rate \(PL\)\. Histograms display relative frequencies computed over all available observations in the dataset\. These variables correspond to longitudinal measurements recorded during patient follow\-up and illustrate the heterogeneous sampling patterns and value ranges typical of real\-world electronic health records\.![Refer to caption](https://arxiv.org/html/2608.16972v1/Supplementary/figures/SUPP-structured-data-boxplots.png)Figure 6:Boxplots of the sequential structured clinical variables used in the study: peripheral oxygen saturation \(PO\), systolic blood pressure \(TA\), Karnofsky performance index \(KAR\), and pulse rate \(PL\)\. The plots summarize the distribution of values across all observations, showing the median, interquartile range, and potential outliers\. These measurements are extracted from longitudinal electronic health records and illustrate the variability and occasional extreme values typical of real\-world clinical data\.VariableCountNaN RatioMeanStdMinQ1MedianQ3MaxPO \(Weight\)5,391,3660\.31666\.3816\.976566677164TA \(Blood Pressure\)2,028,7450\.743118\.8821\.440105117132998KAR \(Karnosky\)4,453,2510\.43580\.2319\.48107080100100PL \(Pulse Rate\)2,019,3310\.74483\.2618\.870718195880Table 3:Descriptive statistics of the sequential structured variables\. The dataset contains7,885,9257\{,\}885\{,\}925observations corresponding to2,5272\{,\}527unique patients\.ModelField / DatasetArchitectureSequential FeaturesC\-indextd\-AUCIBSSigBERT \(\[[21](https://arxiv.org/html/2608.16972#bib.bib21)\]\)Oncology, narrative reports \(Léon Bérard\)OncoBERT \+ Signature \+ Cox LASSOYes, NLP with path signatures0\.750\.80≪\\ll0\.25MSK\-CHORD\[[13](https://arxiv.org/html/2608.16972#bib.bib13)\]Oncology, Real\-world \(MSK\-CHORD\)Random Survival Forest \(RSF\)No, features at fixed time point\[0\.58,0\.83\]\[0\.58,0\.83\]––CoxSig\[[3](https://arxiv.org/html/2608.16972#bib.bib3)\]Maintenance, synthetic \+ real \(NASA, Califrais\)Cox model \+ Signature transformsYes, time\-series encoded with Signature–\[0\.74,0\.87\]\[0\.74,0\.87\]\[0\.09,0\.15\]\[0\.09,0\.15\]BERTSurv\[[27](https://arxiv.org/html/2608.16972#bib.bib27)\]ICU \(MIMIC\-III, not oncology\)Transformer \(BERT\)Yes, from sequential clinical notes \(NLP\)0\.7––DySurv\[[20](https://arxiv.org/html/2608.16972#bib.bib20)\]ICU \(MIMIC\-III, eICU\)CVAE \+ LSTMYes, sequential EHR \(structured\)≈\\approx0\.60–includedSurvival Seq2Seq\[[22](https://arxiv.org/html/2608.16972#bib.bib22)\]General \(MIMIC\-IV \+ synthetic\)Seq2Seq \(GRU\-D \+ Attention\)Yes, hospital time series–\[0\.84,0\.91\]\[0\.84,0\.91\]–Dynamic\-DeepHit\[[17](https://arxiv.org/html/2608.16972#bib.bib17)\]Cystic Fibrosis \(UK Registry\)Deep RNN \+ Temporal AttentionYes, repeated biomarker vectors\[0\.94,0\.96\]\[0\.94,0\.96\]td\-AUC–DeepSurv\[[15](https://arxiv.org/html/2608.16972#bib.bib15)\]General \+ oncology \(e\.g\., METABRIC\)DNN with Cox PH lossNo, static baseline covariates\[0\.61,0\.86\]\[0\.61,0\.86\]––Landmark Endpoint\[[10](https://arxiv.org/html/2608.16972#bib.bib10)\]Liver disease \(PBC\), Aging \(PAQUID\)Landmark \(Cox, RSF, penalized\)Yes, repeated biomarker measures–\[0\.73,0\.87\]\[0\.73,0\.87\]\[0\.076,0\.089\]\[0\.076,0\.089\]Penalized Reg\. Calib\.\[[23](https://arxiv.org/html/2608.16972#bib.bib23)\]Neuromuscular \(DMD, MARK\-MD\)Penalized Cox \+ Mixed EffectsYes, blood biomarker sequences\[0\.7,0\.8\]\[0\.7,0\.8\]\[0\.73,0\.87\]\[0\.73,0\.87\]–

Table 4:Overview of representative survival models across domains\.

Similar Articles

Multi-Modal Machine Learning for Breast Cancer Recurrence Prediction

arXiv cs.LG

This paper examines the integration of multi-modal clinical data, including treatment records, pathology reports, and clinician notes, using rule-based extraction and machine learning to improve breast cancer recurrence prediction compared to single-modal approaches.

Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction

arXiv cs.LG

This paper systematically benchmarks classical machine learning models (Random Forest, XGBoost, etc.) for ER status prediction using multi-omics data from TCGA-BRCA, finding that RNA expression provides the strongest predictive signal and that Random Forest achieves 90.3% balanced accuracy in the integrated multi-omic setting.