PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

# PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models
Source: [https://arxiv.org/html/2606.24102](https://arxiv.org/html/2606.24102)
Lin Lawrence Guo Child Health Evaluative Sciences The Hospital for Sick Children Toronto, Canada &Adam Paul Yan Child Health Evaluative Sciences Division of Haematology/Oncology The Hospital for Sick Children Toronto, CanadaEmily Vettese Child Health Evaluative Sciences The Hospital for Sick Children Toronto, Canada &Lillian Sung Child Health Evaluative Sciences Division of Haematology/Oncology The Hospital for Sick Children Toronto, Canada

###### Abstract

Most electronic health record \(EHR\) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values\. This limits transfer across institutions and even across deployment pipelines within the same institution\. We introduce PORTER \(Portable EHR Representations\), a language\-grounded structured EHR foundation model that decouples event representation from this fixed vocabulary\. PORTER represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines with an autoregressively pretrained temporal backbone\. Across 74 clinical prediction tasks at a pediatric hospital, PORTER matched the mean area\-under\-the\-receiver\-operating\-characteristic curve \(AUROC\) of a fixed\-vocabulary model with the same temporal backbone and pretraining objective\. When the same patient timelines were rendered using event descriptions not seen during pretraining, PORTER transferred without retraining or vocabulary mapping, recovering 97\.1% of the mean AUROC of a model trained directly on the target vocabulary\. When transferred to MIMIC, PORTER outperformed the fixed\-vocabulary model, which dropped 69% of events because their tokens were unseen\. Mechanistic analyses showed cross\-vocabulary transfer tracked preservation of patient\-level representation geometry rather than the scale of the text encoder, and the numeric pathway improved sensitivity to magnitude without disrupting clinical concept identity\. PORTER also achieved higher AUROC than a task\-specific text serialization comparator, at 329\-fold lower amortized compute\. PORTER is a step toward vocabulary\-independent EHR foundation models that reduce the need for vocabulary harmonization while preserving in\-domain performance and enabling efficient cross\-task reuse\.

## 1INTRODUCTION

Structured electronic health record \(EHR\) foundation models learn reusable patient representations through self\-supervised pretraining on longitudinal sequences of clinical events\.\[[39](https://arxiv.org/html/2606.24102#bib.bib39),[36](https://arxiv.org/html/2606.24102#bib.bib36),[44](https://arxiv.org/html/2606.24102#bib.bib44),[34](https://arxiv.org/html/2606.24102#bib.bib34),[38](https://arxiv.org/html/2606.24102#bib.bib38),[37](https://arxiv.org/html/2606.24102#bib.bib37),[8](https://arxiv.org/html/2606.24102#bib.bib8),[31](https://arxiv.org/html/2606.24102#bib.bib31)\]Each event is a timestamped occurrence of a clinical concept, such as a diagnosis or laboratory result, and may include structured attributes such as numeric values\. These models have demonstrated robustness under temporal dataset shift\[[15](https://arxiv.org/html/2606.24102#bib.bib15)\]and transfer across populations\[[26](https://arxiv.org/html/2606.24102#bib.bib26)\]and institutions\[[14](https://arxiv.org/html/2606.24102#bib.bib14)\], although such transfer generally assumes that evaluation data can be encoded using event\-token representations learned during pretraining\.

In practice, the codes and descriptions used to represent clinical events vary across institutions, coding standards, and data pipelines\. Most structured EHR foundation models convert each clinical event into a discrete event token from a fixed vocabulary learned during pretraining\.\[[12](https://arxiv.org/html/2606.24102#bib.bib12)\]These tokens may represent a clinical concept alone or a composite of the clinical concept and structured attributes, such as a discretized numeric bin\.\[[11](https://arxiv.org/html/2606.24102#bib.bib11)\]When deployment data include new clinical concepts, different local codes or names for the same concepts, or new concept\-attribute combinations, those events produce tokens for which the model has no learned representation\. Even within a single institution, models may be developed using retrospective datasets mapped to common data models such as the Observational Medical Outcomes Partnership Common Data Model \(OMOP CDM\), then deployed using production feeds with different local codes or naming conventions\. Therefore, vocabulary shift can arise even when the patient population and clinical setting remain the same\.

A portable structured EHR foundation model must preserve the meaning of a clinical event even when that event is described using different codes or descriptions\. Standard vocabularies and common data models reduce this heterogeneity, but mapping to them is labor\-intensive, error\-prone, and does not eliminate differences in concept coverage or local implementation\. Numeric values introduce a second representation challenge\. For measurement events, the clinical concept identifies what was measured, while the numeric value and reference range determine magnitude and abnormality\. A portable model must therefore represent both concept semantics and numeric values without tying either to a fixed token vocabulary\.

Text\-based representations have emerged as a way to reduce the dependence of structured EHR models on fixed vocabularies and manual harmonization\. One line of work serializes patient timelines into natural language and uses a large language model or text encoder to produce patient\-level representations\.\[[7](https://arxiv.org/html/2606.24102#bib.bib7),[17](https://arxiv.org/html/2606.24102#bib.bib17),[9](https://arxiv.org/html/2606.24102#bib.bib9)\]This approach avoids a fixed event\-token vocabulary, but requires the model to infer concept semantics, temporal relationships, and numeric meaning from the serialized text\.\[[6](https://arxiv.org/html/2606.24102#bib.bib6)\]When serialization is task\-specific, patient representations must also be recomputed for each downstream task\. A second line of work operates at the code or event level by representing clinical concepts through their descriptions or by learning aligned code representations across sites\.\[[25](https://arxiv.org/html/2606.24102#bib.bib25),[20](https://arxiv.org/html/2606.24102#bib.bib20),[48](https://arxiv.org/html/2606.24102#bib.bib48),[2](https://arxiv.org/html/2606.24102#bib.bib2),[21](https://arxiv.org/html/2606.24102#bib.bib21),[19](https://arxiv.org/html/2606.24102#bib.bib19)\]These methods have shown that text\-based code or event representations can improve portability across vocabularies, schemas, institutions, and languages\. They differ in whether the text encoder is frozen or updated during training, which determines how much an event description's representation is shaped by the training vocabulary\. A frozen pretrained encoder applies the same semantic mapping to descriptions from any vocabulary, including vocabularies not seen during training, without modifying model weights\[[25](https://arxiv.org/html/2606.24102#bib.bib25)\]\. Some also use self\-supervised objectives, but as initialization for supervised fine\-tuning rather than to learn task\-agnostic patient representations reused across downstream tasks\.\[[21](https://arxiv.org/html/2606.24102#bib.bib21)\]Across existing frameworks, numeric values are commonly discarded, rendered as text, discretized, or represented through token\-level schemes\. Existing approaches therefore have not fully combined vocabulary\-independent inputs, autoregressive pretraining for reusable task\-agnostic patient representations, and explicit representation of numeric values within a single structured EHR foundation model \(Supplementary Table S1\)\.

We introduce PORTER \(Portable EHR Representations\), a language\-grounded structured EHR foundation model that separates concept semantics, numeric values, and temporal dynamics into distinct components\. Rather than serializing full patient histories into text, PORTER applies language grounding at the clinical event level\. Each event is paired with a natural\-language description of its underlying clinical concept, which is processed by a frozen text encoder to provide vocabulary\-independent concept representations\. Because this text encoder is applied once per unique event description rather than repeatedly across patient histories, these representations can be cached and reused across patients and downstream tasks\. For events containing numeric values, a separate learned pathway encodes numeric magnitude and relative abnormality directly, rather than rendering numeric values as text\. Feature\-wise linear modulation \(FiLM\)\[[33](https://arxiv.org/html/2606.24102#bib.bib33)\]then uses this numeric information to modulate the text\-derived concept representation and produce the event input representation\. Finally, a temporal backbone learns clinical dynamics from sequences of these event input representations through autoregressive pretraining\. After pretraining, the backbone is frozen and produces reusable patient representations that downstream tasks use through linear probes\. At inference, PORTER can represent clinical concepts from vocabularies not seen during pretraining without retraining or explicit vocabulary mapping, provided they can be rendered as interpretable event descriptions\.

We evaluate PORTER across three settings using a fixed\-vocabulary EHR foundation model \(Fixed\-Vocab FM\) matched on backbone architecture, pretraining objective, and training schedule as the primary comparator\. First, we test whether PORTER matches Fixed\-Vocab FM in\-domain across 74 clinical prediction tasks\. Second, we evaluate cross\-vocabulary transfer within the same institution by applying pretrained PORTER to patient timelines where event descriptions are derived from institutional EHR terminology rather than the OMOP\-derived descriptions used during pretraining\. This setting isolates vocabulary shift while holding patient timelines fixed and cannot be directly supported by Fixed\-Vocab FM without vocabulary mapping or retraining\. Third, we evaluate cross\-site transfer to MIMIC, where vocabulary, population, and clinical setting differ from pretraining\. We additionally ablate the numeric pathway and text encoder choice across evaluation settings and compare PORTER with a patient\-level text serialization comparator\. This study makes the following contributions:

- •We introduce PORTER, a language\-grounded structured EHR foundation model that uses event\-description inputs instead of fixed\-vocabulary input embeddings\. PORTER pairs a frozen pretrained text encoder, which yields vocabulary\-independent concept representations that are cached and reused, with a dedicated numeric pathway that integrates magnitude through FiLM, and learns clinical dynamics with an autoregressive temporal backbone that is frozen after pretraining and reused across tasks through linear probes\.
- •We evaluate PORTER using a controlled cross\-vocabulary design that holds patients and downstream task labels fixed while changing only the event\-description naming system\. PORTER transferred to unseen event descriptions without retraining or vocabulary mapping, recovering 97\.1% of the AUROC of a target\-vocabulary reference model\. Text encoder ablations showed that cross\-vocabulary performance tracked preservation of patient\-level representation geometry rather than encoder scale\.
- •PORTER matched an architecture\-matched Fixed\-Vocab FM across 74 in\-domain tasks and improved transfer on 31 of 36 tasks at an external site, where Fixed\-Vocab FM dropped 69% of events because their tokens were unseen\.
- •PORTER’s dedicated numeric pathway improved sensitivity to numeric magnitude compared with rendering values as text, while preserving clinical concept identity\.
- •Compared with a task\-specific patient\-level text serialization comparator, PORTER achieved higher AUROC on 69 of 74 tasks, with lower amortized compute as reuse of task\-agnostic patient representations increased\.

## 2METHODS

### 2\.1Hospital Datasets

This study used EHR data from The Hospital for Sick Children \(SickKids\), a tertiary pediatric hospital, as the primary development site, and MIMIC, derived from Beth Israel Deaconess Medical Center \(BIDMC\), an adult academic medical center, for external evaluation\. SickKids uses Epic \(Epic Systems, Verona, WI\) as its enterprise EHR\.

The SickKids dataset was sourced from the SickKids Enterprise\-wide Data in Azure Repository \(SEDAR\)\[[13](https://arxiv.org/html/2606.24102#bib.bib13)\], which consolidates EHR data from SickKids' Epic Clarity database into a clinically oriented, validated and standardized schema\. EHR data were mapped to the Medical Event Data Standard \(MEDS\)\[[1](https://arxiv.org/html/2606.24102#bib.bib1),[40](https://arxiv.org/html/2606.24102#bib.bib40)\]format with clinical concepts standardized to Observational Medical Outcomes Partnership Common Data Model \(OMOP CDM\) ontologies\. The MIMIC dataset \(MIMIC\-IV, version 1\.0\)\[[23](https://arxiv.org/html/2606.24102#bib.bib23)\]contains de\-identified EHR data from patients admitted to the intensive care unit or emergency department at BIDMC between 2008 and 2019\. MIMIC data were mapped to the OMOP CDM using code provided by the Observational Health Data Sciences and Informatics MIMIC project\[[30](https://arxiv.org/html/2606.24102#bib.bib30)\]and subsequently converted to MEDS format\. As part of MIMIC's de\-identification process, patient timelines are shifted to an anchor year within a three\-year window\. To support consistent temporal splitting across SickKids and MIMIC, we deterministically assigned each patient a representative calendar year within their anchor group via hashing of the patient identifier\.

Use of SEDAR data for this study was approved by the Research Ethics Board \(REB\) at SickKids \(REB number: 1000074527\)\. Use of MIMIC was approved under the oversight of the Institutional Review Board at BIDMC and is made publicly available on PhysioNet\.\[[10](https://arxiv.org/html/2606.24102#bib.bib10)\]

### 2\.2Cohort Definition and Splitting

The cohort selection process is summarized in Supplementary Figure S1\. Pretraining cohorts were defined at the patient level\. For SickKids, we included all patients in SEDAR over the study period, with clinical events spanning June 2, 2018 \(EHR go\-live\) through April 7, 2026\. Patients were excluded if they had missing date of birth\. For MIMIC, all patients in the dataset were included\. Within each dataset, patients were deterministically assigned to training \(~90%\) and validation \(~10%\) subsets via hashing of the patient identifier\. For SickKids, pretraining used events through December 31, 2024 for training patients and through March 31, 2025 for validation patients\. For MIMIC, pretraining used events through December 31, 2016 for training patients and through December 31, 2017 for validation patients\.

Downstream evaluation cohorts were defined at the admission level\. For SickKids, we included inpatient admissions where age at the prediction time was 28 days or older\. For MIMIC, we included inpatient admissions where age at prediction time was 18 years or older\. Admissions were assigned to training, validation, and test sets according to the prediction time, using the same calendar cutoffs applied to the corresponding pretraining cohorts\. Patients with multiple admissions could contribute admissions to different temporal periods, reflecting a deployment setting in which previously observed patients may return for future admissions\.

### 2\.3Clinical Prediction Tasks

The evaluation tasks were adapted from our previous study\.\[[11](https://arxiv.org/html/2606.24102#bib.bib11)\]At SickKids, we evaluated 74 clinical prediction tasks spanning six task families: transfusions, procedures, imaging, laboratory abnormalities, medication administrations, and clinical outcomes, with laboratory abnormalities defined using site\-specific reference ranges\. For MIMIC, we evaluated 36 tasks comprising an adapted subset of the SickKids clinical outcome and laboratory abnormality tasks, with laboratory abnormalities defined using MIMIC\-specific reference ranges\.

Prediction time was set to the end of the admission day at 23:59 for all tasks except 30\-day readmission, for which it was set at 23:59 on the day before discharge\. The prediction window extended until discharge for all tasks except long length of stay and readmission, which used fixed windows of 7 days after admission and 30 days after discharge, respectively\. For each task, admissions in which death, discharge, or outcome occurred on or before the prediction time were excluded\. The full task list and cohort sizes are provided in Supplementary Table S2\.

### 2\.4PORTER Architecture

We define a clinical event as a timestamped EHR occurrence associated with a clinical concept, a natural\-language event description, and, where applicable, numeric metadata such as value and reference range\. Each event contributes one input representation to the temporal model\. In contrast to fixed\-vocabulary models that represent events by indexing a learned embedding table over discrete event tokens, PORTER generates event representations directly from these event descriptions and numeric metadata, making the input pathway vocabulary\-independent\.

PORTER comprises three components \(Figure 1\)\. First, a frozen text encoder converts each event description to a vocabulary\-independent concept representation, which is cached and projected to the transformer hidden dimension\. Second, for events with numeric measurements, a numeric pathway encodes measurement magnitude and relative abnormality, and feature\-wise linear modulation \(FiLM\) integrates this numeric information by scaling and shifting the projected text\-derived representation\. Third, a temporal transformer backbone processes the fused event representations and learns longitudinal clinical dynamics through autoregressive next\-event prediction\. The following sections describe these components and the pretraining objective in detail\.

![Refer to caption](https://arxiv.org/html/2606.24102v1/x1.png)Figure 1:PORTER architecture and evaluation framework\. \(A\) PORTER represents each clinical event through an event\-description pathway and, when applicable, a numeric pathway\. A frozen pretrained text encoder, BioLORD, maps event descriptions to dense embeddings,etexte\_\{\\text\{text\}\}, which are projected into the backbone hidden dimension,htexth\_\{\\text\{text\}\}\. For events with numeric values, a parallel numeric pathway encodes normalized numeric features through a learned multilayer perceptron and modulates the text\-derived concept representation using feature\-wise linear modulation, producing a fused event input representation,hfusedh\_\{\\text\{fused\}\}\. Numeric values are normalized using institutional reference ranges where available, with log\-transformed raw values as a fallback\. Three binary indicators specify whether a numeric value is present and which normalization was applied\. Event\-description embeddings are computed once per unique event description and cached offline\. A causal transformer backbone with 115M parameters and rotary position embeddings applied to patient age in days is pretrained using self\-supervised next\-event prediction over patient event sequences\. The prediction head is discarded after pretraining\. \(B\) Downstream evaluation extracts frozen patient representations at a task\-specific prediction time and trains a task\-specific linear probe\. Three evaluation settings test increasing portability demands\. In\-domain evaluation uses the same institution and OMOP\-derived event descriptions\. Cross\-vocabulary evaluation uses the same patients and labels but SEDAR\-derived event descriptions\. Cross\-site evaluation transfers from SickKids to MIMIC\. A conventional fixed\-vocabulary foundation model, Fixed\-Vocab FM, serves as the primary comparator\. Fixed\-Vocab FM can operate in\-domain and can be evaluated cross\-site when data are harmonized to OMOP, although unseen event tokens are dropped\. It cannot directly support cross\-vocabulary evaluation without vocabulary mapping or retraining\. Abbreviations: PORTER, Portable EHR Representations; MLP, multilayer perceptron; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise\-wide Data in Azure Repository; MIMIC, Medical Information Mart for Intensive Care; FiLM, feature\-wise linear modulation; EHR, electronic health records; RoPE, rotary position embedding\.#### 2\.4\.1Language\-Grounded Event Representation

Each clinical event was represented by a templated event description, with numeric metadata retained for separate numeric encoding\. Templates spanned nine categories: demographics, measurement, observation, condition, procedure, drug, note, specimen, and visit\. Where relevant, templates incorporated additional structured EHR fields, including non\-numeric measurement result categories and drug route\. For example, measurement events were represented as "Measurement: \{concept name\}" or "Measurement: \{concept name\}\. Result: \{non\-numeric result concept name\}\." For the primary PORTER setup, both the primary concept and attribute slots were populated using OMOP concept names\. For cross\-vocabulary evaluation, we constructed a separate set of event descriptions from analogous SEDAR concept and attribute names, allowing the same patient histories to be represented using institutional rather than OMOP terminology\. Full templates and source columns are provided in Supplementary Table S3\. Each event description was encoded once using a frozen text encoder and cached as a static event\-description embedding table\. We evaluated BioLORD\-2023\[[35](https://arxiv.org/html/2606.24102#bib.bib35)\], BGE\-M3\[[4](https://arxiv.org/html/2606.24102#bib.bib4)\], and Qwen3\-Embedding\-8B\[[47](https://arxiv.org/html/2606.24102#bib.bib47)\]as candidate encoders and selected BioLORD\-2023 for the primary PORTER experiments based on the lowest pretraining validation loss \(see Supplementary Figure S2 for pretraining loss curves across text encoder variants\)\. Cached embeddings were projected to the backbone hidden dimension using a two\-layer multilayer perceptron with GELU activation\.

#### 2\.4\.2Numeric Pathway via FiLM

For events with numeric values, PORTER constructs a fixed feature vector capturing measurement magnitude and, when available, relative abnormality without per\-concept population statistics\. When a valid reference range was available, the numeric scalar encoded the value’s position within the range, with 0 corresponding to the lower limit and 1 to the upper limit, clipped to \[\-2, 3\]\. These asymmetric bounds reflect the heavier upper tail of clinical lab values while retaining at least 99% of SickKids training data\. When no reference range was available, PORTER used a log\-magnitude fallback, log\(1 \+ \|v\|\) / 7, clipped to \[0, 1\.5\]\. The divisor of 7 maps log\(1 \+ 1000\)≈\\approx6\.9 to approximately 1\.0\. In SickKids training data, the 99th and 99\.9th percentiles of the resulting pre\-clip scalar were 0\.83 and 1\.32, respectively\. Supplementary Figure S3 provides the empirical distribution of pre\-clip numeric scalar features\.

The numeric feature vector contained this scalar, indicators for reference\-range scaling and log\-magnitude fallback, and an indicator for numeric value presence\. Exactly one scaling indicator was active when a numeric value was present\. For events without numeric content, all indicators and the scalar were set to zero\. This design preserves a fixed numeric feature layout across institutions and vocabularies without relying on concept\-specific distributional summaries, such as means, standard deviations, or quantiles\.

The feature vector was passed through a small multilayer perceptron with two 128\-dimensional fully connected layers and GELU activations\. Two linear heads then produced a residual scaling termγ\\gammaand a shift termβ\\beta, each with dimension equal to the transformer hidden size\. FiLM was applied to the projected text embedding htextas:

hfused=\(1\+γ\)⊙htext\+βh\_\{\\text\{fused\}\}=\(1\+\\gamma\)\\odot h\_\{\\text\{text\}\}\+\\betaTheγ\\gammaandβ\\betaheads were initialized to zero so that the fused representation was initially equal to the text representation\. For events without numeric content, the numeric feature vector was all zeros, and the pathway was gated off by forcingγ\\gamma= 0 andβ\\beta= 0, yielding hfused= htext\.

#### 2\.4\.3Backbone Transformer

The backbone was adapted from the architecture and hyperparameters used in our prior study\[[11](https://arxiv.org/html/2606.24102#bib.bib11)\]\. It was a decoder\-only transformer\[[42](https://arxiv.org/html/2606.24102#bib.bib42),[3](https://arxiv.org/html/2606.24102#bib.bib3)\]with 28 layers, a hidden dimension of 768, and 12 attention heads, yielding 115\.6 million backbone parameters\. Temporal information was encoded using rotary position embeddings \(RoPE\)\[[41](https://arxiv.org/html/2606.24102#bib.bib41)\]applied to patient age in days, so relative positional rotations reflected elapsed time between events\. Transformer layers alternated between global attention and local attention\[[5](https://arxiv.org/html/2606.24102#bib.bib5)\]with a window size of 128 tokens\. Each clinical event contributed one position to the input sequence, corresponding to the event representation passed to the backbone\. For numeric events, this representation was the text\-derived event representation modulated by the numeric pathway, whereas for non\-numeric events, it was the projected text\-derived representation\. Training batches were constructed using event\-budget\-based sequence packing, with a fixed budget of 16,384 clinical events per batch\. Longer patient timelines were truncated, while shorter timelines were packed together using causal masking and patient\-boundary masking to prevent cross\-patient attention\. The maximum effective context window was therefore 16,384 clinical events per patient\. The model was implemented in PyTorch 2\.7\.

#### 2\.4\.4Self\-Supervised Pretraining

PORTER was pretrained on the SickKids training split using autoregressive next\-event prediction, implemented as classification of the next event into a finite output vocabulary of composite discrete tokens observed during pretraining\. Following our prior work\[[11](https://arxiv.org/html/2606.24102#bib.bib11)\], each composite token combined the primary OMOP concept ID with discretized event attributes, including numeric values discretized into 10 quantile bins where applicable\. In PORTER, these composite tokens served only as output labels for autoregressive pretraining, while model inputs were language\-grounded event representations\. The output head was therefore discarded before downstream representation extraction\. Thus, downstream portability depends on the frozen input pathway and temporal backbone, not on reuse of the pretraining output vocabulary\. Training used AdamW\[[27](https://arxiv.org/html/2606.24102#bib.bib27)\]with a peak learning rate of 5×\\times10\-4, cosine decay with warmup, weight decay 0\.05, gradient clipping at 1\.0, Adamβ\\beta1= 0\.9, Adamβ\\beta2= 0\.95, and bfloat16 mixed precision\. Models were trained for five epochs on a single NVIDIA H100 or L40S GPU without early stopping\.\[[28](https://arxiv.org/html/2606.24102#bib.bib28)\]Complete optimizer and training hyperparameters are provided in Supplementary Table S4\.

### 2\.5Comparison Models

We compared PORTER against two comparators: a fixed\-vocabulary foundation model representing the standard discrete\-token input approach\[[39](https://arxiv.org/html/2606.24102#bib.bib39),[36](https://arxiv.org/html/2606.24102#bib.bib36)\], and a patient\-level text serialization comparator representing task\-specific text serialization\.

The Fixed\-Vocab FM served as a controlled comparator that differed from PORTER in how clinical events were represented at input\. Unlike PORTER, where composite tokens served only as output labels during pretraining, Fixed\-Vocab FM used the same composite tokens as both input identifiers and output labels\. Its input layer was therefore a learned embedding table indexed by composite token, and its prediction head predicted the next composite token from the finite pretraining vocabulary\. Pretraining data, backbone transformer, temporal encoding, and training schedule were otherwise matched to PORTER\. During transfer evaluation, Fixed\-Vocab FM could only consume composite tokens observed during SickKids pretraining\. Composite tokens outside this vocabulary were dropped before representation extraction\.

The patient\-level text serialization comparator followed a representative recent approach\.\[[17](https://arxiv.org/html/2606.24102#bib.bib17)\]For each task, the patient timeline up to the task\-specific prediction time was serialized into a structured markdown document containing demographics, recent body metrics, recent vital signs, recent laboratory results, past visits, and visit\-level summaries \(see Supplementary Figure S4 for an illustrative serialized patient timeline\)\. Recent measurements were drawn from curated code lists and limited to the most recent available values\. As in the prior study, each document was truncated to 8,192 input tokens after prepending a task\-specific retrieval instruction \(see Supplementary Table S5 for the full set of task\-specific instructions\)\. The resulting text was encoded with Qwen3\-Embedding\-8B using last\-token pooling to produce a single 4,096\-dimensional task\-conditioned patient representation\. Because the retrieval instruction was task\-specific, patient\-level embeddings were generated separately for each task\. In contrast, PORTER and Fixed\-Vocab FM produced task\-agnostic patient representations that were reused across downstream tasks\. We therefore used the patient\-level text serialization comparator only in in\-domain evaluation, not in cross\-vocabulary or cross\-site evaluations\.

### 2\.6Ablation Experiments

We conducted two sets of ablation experiments to evaluate key architectural decisions, with all variants pretrained on the same SickKids training split using the same backbone, pretraining objective, optimizer, and training schedule\.

#### 2\.6\.1Text Encoder Ablation

We compared three frozen text encoders: BioLORD\-2023 \(768\-dimensional\), BGE\-M3 \(1,024\-dimensional\), and Qwen3\-Embedding\-8B \(4,096\-dimensional\), plus a non\-semantic random\-embedding baseline\. For the random baseline, each unique rendered event\-description string was assigned a fixed 4,096\-dimensional vector drawn from a standard normal distribution and L2\-normalized to unit length\. Vectors were deterministically keyed by the rendered event\-description string, so identical event\-description strings received identical vectors across vocabularies, whereas semantically equivalent but textually distinct event\-description strings received uncorrelated vectors\. This baseline preserves event\-description identity while removing semantic structure\. The lookup table was cached in the same manner as the text\-encoder embeddings\. Because embedding dimensionality differed across encoders, the input dimension of the text projection MLP \(and consequently its parameter count\) also varied, while the templated event descriptions and numeric pathway remained identical\.

#### 2\.6\.2Numeric Encoding Ablation

We compared three architectural strategies for integrating numeric values into the language\-grounded event representation: 1\) PORTER, which encodes numeric values through the numeric pathway and integrates them with the text\-derived event representation through FiLM; 2\) PORTER\-NoNum, which removes the numeric pathway and FiLM entirely and uses only the text\-derived representation; and 3\) PORTER\-NumText, which removes the numeric pathway and FiLM but incorporates numeric values, units, and reference ranges directly into the templated event descriptions before encoding, testing whether the frozen text encoder alone could represent numeric magnitude when numeric information was expressed in language\.

### 2\.7Downstream Evaluation

#### 2\.7\.1Full\-Shot Evaluation

After pretraining, the backbone was frozen and hidden states were extracted at prediction time for each of the 74 downstream binary classification tasks\. Linear probes were trained using L2\-regularized logistic regression implemented in Scikit\-learn\[[32](https://arxiv.org/html/2606.24102#bib.bib32)\], with features standardized to zero mean and unit variance using statistics computed on the training subset of each task's split\. Models were fit with LBFGS for up to 10,000 iterations, with inverse regularization strength selected from \{1, 10\-1, 10\-2, 10\-3, 10\-4\} on the validation set\. Performance was reported as AUROC on the held\-out temporal test split\.

#### 2\.7\.2Sample Efficiency Evaluation

To assess sample efficiency, models were evaluated across labeled training set sizes \(k\) ranging from 2 to 32,768 examples, in powers of two\. For each shot size, 10 independent iterations were conducted using a fixed sequence of random seeds applied identically across tasks\. In each iteration, k training examples were drawn from the task\-specific training set using balanced sampling \(equal positive and negative examples where possible\), while the validation and test sets remained unchanged\. For tasks with fewer than k/2 positive instances available, all available positive instances were included, with the remaining examples drawn from the negative class\. The same linear probe protocol was used\. Task performance was reported as the mean across the 10 iterations for each shot size\.

#### 2\.7\.3Cross\-Vocabulary Evaluation

To isolate vocabulary shift from differences in patient population, outcome definitions, and temporal evaluation period, the cross\-vocabulary evaluation used the same underlying SickKids admissions, prediction times, labels, and downstream splits as the in\-domain evaluation\. Only the source naming system used to render event descriptions was changed\. In the primary PORTER setup, OMOP\-derived event descriptions were constructed from OMOP concept names\. In the cross\-vocabulary setup, native SEDAR\-derived event descriptions for the same patient timelines were regenerated using SEDAR concept names\. For PORTER, the cached text\-embedding table was regenerated from these alternate event descriptions using the same frozen text encoder, while the projection layer, FiLM module, and temporal backbone remained unchanged and frozen\. This tested whether a PORTER model pretrained with OMOP\-derived descriptions could consume the same clinical timelines rendered using a source naming system not seen during pretraining, without retraining the input pathway or backbone\. Fixed\-Vocab FM could not be evaluated in this setting because its input embedding table was indexed by composite event tokens observed during pretraining, and SEDAR composite event tokens had no learned input embeddings\. Downstream evaluation followed the same linear\-probe full\-shot and sample\-efficiency evaluation protocols used for in\-domain evaluation\.

To provide an upper\-bound reference, we trained a Fixed\-Vocab FM from scratch on SickKids data using composite event tokens constructed from native SEDAR event\-attribute combinations, and evaluated it under the same full\-shot protocol\.

#### 2\.7\.4Cross\-Site Evaluation

Cross\-site generalization was evaluated by applying models pretrained on SickKids to MIMIC\. For PORTER, MIMIC OMOP\-derived event descriptions were rendered using the same templates, with slots populated from MIMIC OMOP concept names, then encoded with the same frozen text encoder to construct a MIMIC\-specific cached embedding table\. The SickKids\-trained projection layer, FiLM module, and temporal backbone were reused unchanged\. Thus, PORTER adapted its input cache to the MIMIC OMOP\-derived description set through text encoding, without modifying learned model weights before downstream evaluation\.

Fixed\-Vocab FM was transferred by applying its SickKids\-trained composite\-token vocabulary to MIMIC composite tokens\. MIMIC events whose composite token was not observed during SickKids pretraining were dropped before representation extraction\. We quantified event\-drop rates as the fraction of MIMIC events removed by this procedure\. In contrast, PORTER does not drop any events: any MIMIC event whose event description the frozen text encoder can read produces a representation\.

Downstream evaluation for PORTER and Fixed\-Vocab FM followed the same linear\-probe full\-shot and sample\-efficiency evaluation protocols used for in\-domain evaluation\.

To provide an upper\-bound reference, we trained a Fixed\-Vocab FM from scratch on MIMIC data using the same pretraining procedure and evaluated it under the full\-shot protocol\.

### 2\.8Mechanistic Analyses

We performed representation\-level analyses to characterize mechanisms underlying cross\-vocabulary transfer and numeric encoding\. To assess whether cross\-vocabulary transfer preserved patient\-level geometry, we extracted test\-set patient representations from PORTER variants with different text encoders after rendering the same patient timelines under two source naming systems: OMOP\-derived descriptions and native SEDAR\-derived descriptions\. The conditions differed only in the names used to construct event descriptions, whereas the underlying events, timestamps, and numeric metadata were identical\. We computed pairwise cosine distances among test patients under each condition\. Relative geometry was quantified as the Spearman correlation between the OMOP\-derived and SEDAR\-derived distance matrices\. Preservation of distance scale was summarized by ordinary least\-squares regression of SEDAR\-derived distances on OMOP\-derived distances, with slope near 1 and intercept near 0 indicating minimal compression, expansion, or shift across naming systems\.

At the event level, we evaluated synonym invariance and numeric sensitivity of the event representations passed to the transformer for PORTER, PORTER\-NoNum, and PORTER\-NumText\. Synonym invariance was assessed using OMOP\-derived and SEDAR\-derived event\-description pairs across drug, measurement, procedure, condition, and observation events\. Because multiple SEDAR concept names could correspond to the same OMOP concept, this analysis tested whether SEDAR\-derived naming variants were mapped near their shared OMOP\-derived reference\. We retained concepts with at least two distinct SEDAR\-derived event descriptions and excluded queries in which the OMOP concept name appeared within the SEDAR\-derived description\. For each concept, up to 10 SEDAR\-derived descriptions were used as queries\. The matched OMOP\-derived description served as the within\-concept reference, and 32 sampled OMOP\-derived descriptions from other concepts in the same domain served as distractors\. Cosine similarity was compared between query\-reference and query\-distractor pairs\.

Numeric sensitivity was assessed among measurement concepts with at least 1,000 numeric values recorded in a common unit\. For each concept, we constructed the event representation passed to the transformer at empirical deciles of the observed value distribution for PORTER, PORTER\-NoNum, and PORTER\-NumText\. Representations were L2\-normalized\. For each concept, cosine distance from the first\-decile representation to each subsequent decile representation was computed, and Spearman correlation between decile rank and distance was measured\. Higher positive correlation indicated stronger monotonic sensitivity of the representation to numeric magnitude\. PORTER\-NoNum was invariant to decile by construction, so its correlation was undefined\.

### 2\.9Compute Cost Analysis

Total compute for PORTER was estimated as the sum of backbone pretraining floating\-point operations \(FLOPs\), one\-time event description text\-embedding FLOPs, and backbone extraction FLOPs for generating patient representations over the downstream task cohorts\. Backbone pretraining was estimated using the standard 6ND\[[24](https://arxiv.org/html/2606.24102#bib.bib24)\]approximation, where N is the number of non\-embedding backbone parameters and D is the number of EHR event tokens processed during training\. Event description text embeddings were computed once over unique event descriptions and estimated as 2NtextDtextforward\-pass FLOPs through BioLORD, where Ntextis the number of non\-embedding text\-encoder parameters and Dtextis the total number of event\-description tokens\. Backbone extraction was estimated as the forward\-pass FLOPs of the pretrained EHR backbone over patient records in the task cohort\. For Fixed\-Vocab FM, only backbone pretraining and backbone extraction FLOPs applied\.

For the text serialization comparator, per\-task compute was estimated as Qwen3\-Embedding\-8B forward\-pass FLOPs summed over all patient records in that task’s cohort\. Because the serialized input and retrieval instruction were task\-specific, representations could not be shared across tasks, so total compute scaled with the number of downstream tasks\. We report aggregate compute across the 74 SickKids tasks and its effective scaling factor relative to PORTER\.

### 2\.10Statistical Analysis

Model comparisons were based on paired per\-task AUROC values\. For each model pair, we computed per\-task AUROC differences across the 74 SickKids tasks for in\-domain and cross\-vocabulary evaluations, or the 36 MIMIC tasks for cross\-site evaluation, and tested whether the paired differences were symmetrically centered around zero using a two\-sided Wilcoxon signed\-rank test\[[45](https://arxiv.org/html/2606.24102#bib.bib45)\]implemented in SciPy\.\[[43](https://arxiv.org/html/2606.24102#bib.bib43)\]When multiple comparisons were performed within an ablation family \(text encoder choice or numeric encoding strategy\), p\-values were adjusted using Holm's step\-down procedure\[[18](https://arxiv.org/html/2606.24102#bib.bib18)\]to control the family\-wise error rate atα\\alpha= 0\.05\. Holm correction was applied separately within each evaluation setting: in\-domain, cross\-vocabulary, and cross\-site\.

## 3RESULTS

### 3\.1Study Cohorts, Vocabularies, and Pretraining Compute

The SickKids and MIMIC pretraining cohorts comprised 2,100,646 and 339,989 patients, with 220\.3 and 181\.3 million clinical events, respectively \(Table 1\)\. Downstream evaluation included 101,404 SickKids admissions and 58,513 MIMIC admissions\. The SickKids OMOP, SEDAR, and MIMIC OMOP source concept sets included 15,516, 105,551, and 32,032 unique clinical concepts, with 19,686, 523,724, and 36,524 unique rendered event\-descriptions, respectively\. SickKids OMOP and MIMIC OMOP shared 1,153 event\-descriptions, whereas SickKids OMOP and SEDAR shared only 2\. Across models, pretraining compute was similar at approximately 1\.0×\\times1018FLOPs\. Trainable input\-pathway parameters ranged from 1\.18M for PORTER variants without numeric integration to 75\.5M for the Fixed\-Vocab FM learned vocabulary embedding table \(Supplementary Table S6\)\.

Table 1:Characteristics of pretraining and task cohorts\.aPretraining cohorts include patients and events used for self\-supervised pretraining of the SickKids and the reference MIMIC foundation models\.

bTask cohorts include admissions considered for downstream clinical prediction evaluation\. The MIMIC cohort was also used for the external evaluation of the SickKids foundation models\.

Abbreviations: IQR, interquartile range; SickKids, The Hospital for Sick Children; MIMIC, Medical Information Mart for Intensive Care\.

### 3\.2PORTER Matches Fixed\-Vocab FM In\-Domain and Improves Transfer Across Vocabularies and Institutions

On the 74 SickKids in\-domain tasks, PORTER and Fixed\-Vocab FM did not differ in mean AUROC \(0\.884 vs 0\.884, p=0\.942; Table 2, Supplementary Table S7\)\. In the cross\-vocabulary setting, SickKids timelines were re\-expressed using SEDAR\-derived event descriptions\. PORTER achieved a mean AUROC of 0\.848 without retraining or vocabulary mapping, recovering 97\.1% of the performance of an upper\-bound reference model trained directly on SEDAR composite event tokens \(mean AUROC, 0\.873\)\. Fixed\-Vocab FM could not be directly evaluated in this setting because its learned input vocabulary did not contain SEDAR composite event tokens\. On cross\-site evaluation in MIMIC, PORTER had higher AUROC than Fixed\-Vocab FM on 31 of 36 tasks \(mean AUROC 0\.823 vs 0\.810, p<0\.001\)\. For Fixed\-Vocab FM, 69% of MIMIC events contained unseen composite tokens and were dropped during representation extraction, whereas PORTER produced representations for all MIMIC events\. An upper\-bound reference model trained directly on MIMIC achieved mean AUROC of 0\.848\. Across labeled training\-set sizes, PORTER and Fixed\-Vocab FM were similar in\-domain at larger sample sizes, while PORTER was higher in few\-shot in\-domain settings and across sample sizes in MIMIC \(Supplementary Figure S5\)\.

Table 2:In\-domain, cross\-vocabulary, and cross\-site mean AUROC of PORTER and Fixed\-Vocab FM\.Bold: the statistical reference model\.

\*\*\*p<0\.001p<0\.001vs PORTER \(reference\) using two\-sided Wilcoxon signed\-rank test\.

aFixed\-Vocab FM was not evaluated in cross\-vocabulary transfer because SEDAR composite event tokens had no learned input embeddings\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; PORTER, Portable EHR Representations; EHR, electronic health records; FM, foundation model; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise\-wide Data in Azure Repository\.

### 3\.3PORTER Outperforms a Task\-Specific Text Serialization Comparator on AUROC, Sample Efficiency, and Amortized Compute

PORTER had higher full\-shot AUROC than the patient\-level text serialization comparator on 69 of 74 in\-domain tasks \(mean AUROC 0\.884 vs 0\.863, p<0\.001; Figure 2A, Supplementary Table S8\)\. Across labeled training\-set sizes, PORTER had higher AUROC from 4 examples per task onward, with larger differences between 32 to 512 examples per task \(Figure 2B\)\. The text serialization comparator required less compute when only a small number of patient\-task representations were generated, whereas PORTER required less compute once representations were generated for more than approximately 256 admissions per task, and required 329\-fold fewer amortized FLOPs across all 74 task cohorts \(Figure 2C\)\.

![Refer to caption](https://arxiv.org/html/2606.24102v1/x2.png)Figure 2:Performance, sample efficiency, and compute for PORTER versus the patient\-level text serialization comparator\. \(A\) Per\-task AUROC difference between PORTER and the patient\-level text serialization comparator \(Qwen3\-Embedding\-8B with patient\-level text serialization\) across 74 SickKids clinical prediction tasks evaluated in\-domain\. Each point represents one task\. Blue points indicate tasks where PORTER performed better, and red points indicate tasks where the text serialization comparator performed better\. Tasks are grouped by clinical category, with per\-category win counts annotated\. \(B\) Mean AUROC difference between PORTER and the text serialization comparator, left axis, and mean AUROC for each approach, right axis, as a function of labeled training examples per task\. Shaded region indicates±1\\pm 1standard error of the mean difference across tasks\. \(C\) Percentage reduction in total floating\-point operations achieved by PORTER relative to the text serialization comparator, left axis, alongside total FLOPs for each approach, right axis\. PORTER patient representations are computed once and reused across tasks, whereas the text serialization comparator requires a separate forward pass through Qwen3\-Embedding\-8B for each patient\-task pair\. As a result, comparator compute scales with the number of downstream tasks, while PORTER’s amortized compute advantage increases with downstream reuse\. Abbreviations: PORTER, Portable EHR Representations; EHR, electronic health records; AUROC, area under the receiver operating characteristic curve; FLOPs, floating\-point operations\.
### 3\.4Text Encoder Choice Determines Cross\-Vocabulary Transfer More Than In\-Domain Performance

Holding the numeric pathway and FiLM fixed, encoder\-related AUROC differences were largest in the cross\-vocabulary setting, where SickKids OMOP and SEDAR shared almost no event\-description strings \(Table 3, Supplementary Table S9\)\. In\-domain differences were statistically significant but small: 0\.884 for BioLORD, 0\.881 for Qwen3, 0\.881 for BGE\-M3, and 0\.879 for Random \(all p<0\.001 vs BioLORD\)\. Cross\-vocabulary differences were larger: 0\.848, 0\.840, 0\.818, and 0\.780, respectively, with the largest degradation observed for Random \(all p<0\.001 vs BioLORD\)\. On cross\-site evaluation, Qwen3 \(0\.825\) and BGE\-M3 \(0\.821\) were statistically indistinguishable from BioLORD \(0\.823\), while Random was lower \(0\.812; p<0\.001\)\. Sample\-efficiency curves showed limited in\-domain differences but larger cross\-vocabulary and cross\-site performance deficits relative to BioLORD across labeled sample sizes, again most prominently for Random \(Supplementary Figure S6\)\.

Table 3:Effect of text encoder choice on in\-domain, cross\-vocabulary, and cross\-site mean AUROC\.Bold: the statistical reference model\.

Significance vs BioLORD \(reference\);\*\*\*p<0\.001p<0\.001\(two\-sided Wilcoxon signed\-rank test, Holm\-corrected across 3 reference comparisons per setting\)\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care; B, billion\.

Preservation of patient\-level representation geometry between OMOP\-derived and SEDAR\-derived event descriptions followed the same across\-encoder ranking as cross\-vocabulary AUROC\. Pairwise cosine distances between test patients computed under the two description sources had Spearman correlations of 0\.613 for BioLORD, 0\.606 for Qwen3, 0\.515 for BGE\-M3, and 0\.267 for Random\. Distance\-scale preservation showed the same ranking, with OLS slopes/intercepts of 0\.660/0\.018, 0\.602/0\.086, 0\.478/0\.137, and 0\.318/0\.210, respectively \(Supplementary Figure S7\)\.

### 3\.5FiLM Numeric Pathway Improves Numeric Sensitivity Without Disrupting Concept Identity

Holding the text encoder fixed, mean AUROC differed by numeric encoding strategy \(Table 4, Supplementary Table S10\)\. PORTER had the highest mean AUROC in all three settings: 0\.884 in\-domain versus 0\.880 for PORTER\-NumText and 0\.873 for PORTER\-NoNum; 0\.848 cross\-vocabulary versus 0\.839 and 0\.837; and 0\.823 cross\-site versus 0\.801 and 0\.804, respectively \(all p<0\.001 vs PORTER\)\. Sample\-efficiency curves followed the same overall pattern \(Supplementary Figure S8\)\.

Table 4:Effect of numeric encoding strategy on in\-domain, cross\-vocabulary, and cross\-site mean AUROC\.Bold: the statistical reference model\.

Significance vs PORTER \(reference\);\*\*\*p<0\.001p<0\.001\(two\-sided Wilcoxon signed\-rank test, Holm\-corrected across 2 reference comparisons per setting\)\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; PORTER, Portable EHR Representations; EHR, electronic health records; NoNum, no numeric pathway \(text\-derived representations only\); NumText, numeric values rendered as text; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care\. MIMIC cross\-site results include 36 tasks\.

Analyses of the event input representations passed to the transformer showed that all three variants preserved clinical concept identity, with within\-concept cosine similarity exceeding across\-concept similarity across event domains \(Supplementary Figure S9\)\. For numeric sensitivity, the Spearman correlation for each measurement concept between value\-decile rank and cosine distance from the first\-decile representation had median 1\.00 for PORTER and 0\.67 for PORTER\-NumText\. PORTER\-NoNum was invariant to numeric value by construction, so this correlation was undefined\.

## 4DISCUSSION

PORTER enabled direct cross\-vocabulary transfer to patient timelines expressed with event descriptions not used during pretraining, achieving 97\.1% of the AUROC of a reference model trained directly on the target vocabulary\. This transfer required neither PORTER retraining nor vocabulary mapping\. PORTER also matched a fixed\-vocabulary foundation model across 74 in\-domain tasks and improved transfer on 31 of 36 tasks at an external site\. Compared with a task\-specific patient\-level text serialization comparator, PORTER achieved higher AUROC on 69 of 74 in\-domain tasks and showed an amortized compute advantage that increased with the number of downstream tasks and patients, reaching 329\-fold in the full evaluation\. Ablations clarified the distinct roles of PORTER’s architectural components\. The temporal backbone preserved in\-domain performance across input representations\. Cross\-vocabulary performance tracked how well each encoder preserved patient\-level representation geometry across vocabularies, an emergent property of the pretrained representations that is not explained by encoder scale alone\. The dedicated numeric pathway improved sensitivity to numeric magnitude without degrading clinical concept identity\.

PORTER shows that vocabulary\-independent inputs can support self\-supervised EHR foundation model pretraining without compromising task\-agnostic representation learning\. Prior work has shown that encoding clinical concepts or events through text can reduce dependence on fixed vocabularies and improve transfer across institutions and languages\.\[[25](https://arxiv.org/html/2606.24102#bib.bib25),[20](https://arxiv.org/html/2606.24102#bib.bib20),[21](https://arxiv.org/html/2606.24102#bib.bib21),[19](https://arxiv.org/html/2606.24102#bib.bib19)\]However, these approaches generally use text\-based representations within supervised settings\. Even when self\-supervised objectives are used, the resulting model is typically adapted through downstream fine\-tuning rather than used as a frozen source of task\-agnostic patient representations\. PORTER instead uses autoregressive pretraining to learn a temporal backbone that is frozen and reused across downstream tasks through linear probes\. The cross\-vocabulary evaluation isolates vocabulary shift from the population and practice differences that confound cross\-site comparisons\. This setting also reflects a common deployment challenge in which models trained on retrospective research datasets may later be applied to production feeds that use different vocabularies or naming conventions\. Fixed\-vocabulary foundation models lack a direct mechanism for this setting without vocabulary mapping or retraining\. Together with the in\-domain parity result, these findings support vocabulary\-independent input representations as a viable design choice for structured EHR foundation models\.

Ablation experiments clarified the distinct roles of PORTER’s architectural components\. Under in\-domain evaluation, random embeddings preserved much of PORTER’s AUROC\. This indicates that the temporal backbone can learn useful clinical dynamics when clinical events are represented consistently, even without semantic structure in the input vectors\. In cross\-vocabulary evaluation, however, random embeddings degraded performance substantially, likely because distinct descriptions of the same clinical concept were assigned unrelated embeddings\. Across PORTER variants using different text encoders, cross\-vocabulary AUROC tracked the extent to which each encoder preserved patient\-level pairwise distance structure across vocabularies\. Next\-event prediction does not optimize for this geometry, so this correspondence is an emergent property of the pretrained representations and points to semantic alignment as the basis for vocabulary transfer\. Finally, using the dedicated numeric pathway increased ordinal sensitivity from 0\.67 to 1\.00 compared to text\-encoded numeric values while preserving within\-concept representation structure\. This indicates that FiLM\-based numeric integration strengthened numeric sensitivity in the event input representation without disrupting the concept semantics supplied by the event\-description embedding\. The advantage of the dedicated numeric pathway over rendering numeric values as text widened in cross\-vocabulary and cross\-site evaluations, consistent with normalized numeric features transferring more reliably than text\-rendered magnitudes\. In PORTER, this normalization is reference\-range anchored when available and uses a bounded log\-magnitude fallback otherwise\. Together, these findings indicate that PORTER's components serve distinct and complementary roles and can in principle be improved independently\.

The encoder ablation findings suggest that text\-encoder scale is not the primary determinant of PORTER’s performance\. Across evaluations, BioLORD, a 109M\-parameter biomedical encoder trained to align biomedical concept names with ontology\-informed definitions, performed on par with or better than general\-purpose encoders up to 63 times larger, with the largest advantage in cross\-vocabulary evaluation\. This pattern may reflect PORTER’s narrower requirement for the text encoder, which is to embed short clinical event descriptions so that alternative descriptions of the same clinical concepts remain close across vocabularies\. By contrast, Qwen3\-Embedding and BGE\-M3 are general\-purpose embedding models optimized across broad multilingual, retrieval, relevance\-ranking, and long\-context settings\. Standard embedding benchmarks evaluate many useful capabilities\[[29](https://arxiv.org/html/2606.24102#bib.bib29)\], but they do not directly measure the properties most relevant to language\-grounded EHR event modeling, including preservation of patient\-level geometry across vocabularies and ordinal sensitivity to numeric values\. Developing encoder benchmarks around these properties could make encoder selection more efficient and may improve cross\-vocabulary transfer without changing the temporal backbone or numeric pathway\.

The text encoder findings also inform the comparison between PORTER and patient\-level text serialization\. In PORTER, language grounding occurs at the event level\. Each clinical event is represented using a frozen text encoder and dedicated numeric pathway, and the temporal backbone integrates sequences of event input representations into reusable patient representations\. Patient\-level serialization instead represents the full patient timeline as text and encodes it directly as a patient\-level representation, often within a task\-specific prompt\. This provides flexible task conditioning but requires the model to represent the full task\-conditioned patient timeline\. Prior patient\-level serialization work suggests that this setting favors larger general\-purpose encoders\.\[[17](https://arxiv.org/html/2606.24102#bib.bib17)\]The compute profiles also differ\. PORTER’s event\-description embeddings are computed once per unique description, and the resulting patient representations can be reused across downstream tasks\. In contrast, the task\-specific serialization comparator we evaluate requires a language\-model forward pass for each patient\-task pair\. Recent rubric\-based serialization methods synthesize an extraction schema from labeled examples and report higher accuracy than direct serialization\[[7](https://arxiv.org/html/2606.24102#bib.bib7)\], although they still build patient representations tied to a specific task\. These differences suggest that PORTER suits high\-throughput, population\-scale prediction, whereas patient\-level serialization may suit settings that value flexible, task\-specific conditioning over reuse\.

Several limitations should be noted\. All models in the main experiments were pretrained on data from a single pediatric hospital, and cross\-site evaluation was limited to one external adult intensive care setting, where vocabulary mismatch and population shift co\-occur\. All downstream evaluations used linear probes on frozen representations for binary classification tasks\. Performance under fine\-tuning, more expressive adaptation methods\[[16](https://arxiv.org/html/2606.24102#bib.bib16),[46](https://arxiv.org/html/2606.24102#bib.bib46)\], regression tasks, zero\-shot prediction, or generative settings has not been assessed\. We also kept the text encoder frozen and did not test fine\-tuning it end\-to\-end during pretraining, which could change which encoder properties matter\.\[[19](https://arxiv.org/html/2606.24102#bib.bib19)\]PORTER depends on the availability and quality of event descriptions and numeric metadata, including reference ranges, which may vary across institutions and data pipelines\. Finally, although PORTER removes fixed\-vocabulary dependence on the input side, its autoregressive pretraining objective still uses a fixed output vocabulary for next\-event prediction\. The language\-grounded input pathway is independent of this objective and could be paired with alternatives such as time\-to\-event or continuous\-time generative formulations\.\[[38](https://arxiv.org/html/2606.24102#bib.bib38),[37](https://arxiv.org/html/2606.24102#bib.bib37),[8](https://arxiv.org/html/2606.24102#bib.bib8),[22](https://arxiv.org/html/2606.24102#bib.bib22)\]Because these alternatives are also typically defined over a fixed set of output codes, reducing output\-side vocabulary dependence remains future work\.

In conclusion, PORTER shows that language\-grounded event representations can make structured EHR foundation models portable across vocabularies while preserving in\-domain performance and allowing patient representations to be reused efficiently across downstream tasks\. By modeling concept semantics, numeric values, and temporal dynamics separately, PORTER reduces dependence on fixed vocabularies and provides a practical route toward EHR foundation models that can generalize across deployment settings and institutions\.

## 5Data Availability

The SickKids dataset cannot be made publicly available due to patient privacy restrictions\. Relevant data are available upon reasonable request to the corresponding author\. The MIMIC dataset is publicly available through PhysioNet \(https://physionet\.org/content/mimiciv/1\.0/\) subject to credentialing and a data use agreement\.

## 6Code Availability

## 7Acknowledgements

LS is supported by the Canada Research Chair in Pediatric Oncology Supportive Care\.

We thank Jason Alan Fries and Natalie Pageler for their helpful feedback on an earlier draft of this manuscript\.

## 8Funding

This research did not receive funding\.

## 9Author Contribution

L\.L\.G\. conceptualized and designed the study with input from all authors\. L\.L\.G\. performed all experiments, analyzed and interpreted results with input from all authors\. L\.L\.G\. wrote the manuscript with input from all authors\. L\.L\.G, A\.P\.Y, E\.V, and L\.S read and approved the final manuscript\.

## 10Competing Interests

The authors declare no competing interests\.

## References

- \[1\]B\. Arnrich, E\. Choi, J\. A\. Fries, M\. B\. A\. McDermott, J\. Oh, T\. Pollard, N\. Shah, E\. Steinberg, M\. Wornow, and R\. van de Water\(2024\)Medical event data standard \(MEDS\): facilitating machine learning for health\.Conference ProceedingsInICLR 2024 Workshop on Learning from Time Series For Health,External Links:[Link](https://openreview.net/forum?id=IsHy2ebjIG)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p2.1)\.
- \[2\]R\. A\. Attrach, R\. Fani, D\. Restrepo, Y\. Jia, and P\. Schüffler\(2025\)Rethinking tokenization for clinical time series: when less is more\.arXiv2512\.05217\.External Links:[Link](https://arxiv.org/abs/2512.05217)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1)\.
- \[3\]T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei\(2020\)Language models are few\-shot learners\.Advances in Neural Information Processing Systems33,pp\. 1877–1901\.External Links:[Link](https://papers.nips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html)Cited by:[§2\.4\.3](https://arxiv.org/html/2606.24102#S2.SS4.SSS3.p1.1)\.
- \[4\]J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. Liu\(2024\)M3\-Embedding: multi\-linguality, multi\-functionality, multi\-granularity text embeddings through self\-knowledge distillation\.Findings of the Association for Computational Linguistics: ACL 2024,pp\. 2318–2335\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.137),[Link](https://aclanthology.org/2024.findings-acl.137/)Cited by:[§2\.4\.1](https://arxiv.org/html/2606.24102#S2.SS4.SSS1.p1.1)\.
- \[5\]R\. Child, S\. Gray, A\. Radford, and I\. Sutskever\(2019\)Generating long sequences with sparse transformers\.arXiv1904\.10509\.External Links:[Link](https://arxiv.org/abs/1904.10509)Cited by:[§2\.4\.3](https://arxiv.org/html/2606.24102#S2.SS4.SSS3.p1.1)\.
- \[6\]H\. Cui, A\. Unell, B\. Chen, J\. A\. Fries, E\. Alsentzer, S\. Koyejo, and N\. H\. Shah\(2025\)TIMER: temporal instruction modeling and evaluation for longitudinal clinical records\.npj Digital Medicine8\(1\),pp\. 577\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-025-01965-9),[Link](https://doi.org/10.1038/s41746-025-01965-9)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1)\.
- \[7\]I\. Demirel, L\. Shi, Z\. Hussain, and D\. Sontag\(2026\)LLMs can construct powerful representations and streamline sample\-efficient supervised learning\.arXiv2603\.11679\.External Links:[Link](https://arxiv.org/abs/2603.11679)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§4](https://arxiv.org/html/2606.24102#S4.p5.1)\.
- \[8\]C\. Gadd, K\. Gokhale, A\. Acharya, J\. Cooper, F\. Crowe, L\. Fitzsimmons, T\. Jackson, K\. Nirantharakumar, C\. Yau, R\. Birch, M\. Canducci, D\. Danks, A\. d’Elia, A\. Denniston, S\. Flanagan, S\. Gallier, N\. Guan, X\. Guan, I\. Guellil, G\. Gkoutos, S\. Haroon, E\. Hathaway, L\. Jackson, J\. Lord, Z\. Majid, T\. Marshall, G\. Morris, C\. Owen, E\. Sapey, C\. Sainsbury, C\. Spurway, P\. Tino, S\. Wambua, A\. Azcoaga\-Lorenzo, C\. McCowan, L\. R\. Pedro, M\. Usman, N\. Hong, S\. Matijevic, K\. Martens, T\. Williams, P\. Myles, and The OPTIMAL Consortium\(2026\)SurvivEHR: a competing risks, time\-to\-event foundation model for multiple long\-term conditions from primary care electronic health records\.npj Digital Medicine\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-026-02709-z),[Link](https://doi.org/10.1038/s41746-026-02709-z)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1),[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[9\]J\. Gao, M\. Rahman, J\. Caskey, M\. Oguss, A\. O’Rourke, R\. Brown, A\. Stey, A\. Mayampurath, M\. M\. Churpek, G\. Chen, and M\. Afshar\(2026\)MoMA: a mixture\-of\-multimodal\-agents architecture for enhancing clinical prediction modelling\.npj Digital Medicine9\(1\),pp\. 46\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-025-02219-4),[Link](https://doi.org/10.1038/s41746-025-02219-4)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1)\.
- \[10\]A\. L\. Goldberger, L\. A\. N\. Amaral, L\. Glass, J\. M\. Hausdorff, P\. Ch\. Ivanov, R\. G\. Mark, J\. E\. Mietus, G\. B\. Moody, C\. Peng, and H\. E\. Stanley\(2000\)PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals\.Circulation101\(23\),pp\. e215–e220\.External Links:[Document](https://dx.doi.org/10.1161/01.CIR.101.23.e215),[Link](https://www.ahajournals.org/doi/abs/10.1161/01.CIR.101.23.e215)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p3.1)\.
- \[11\]L\. L\. Guo, S\. E\. Arciniegas, J\. J\. Lee, A\. P\. Yan, G\. Tomlinson, J\. Fries, and L\. Sung\(2026\)Tokenization tradeoffs in structured EHR foundation models\.arXiv2603\.15644\.External Links:[Link](https://arxiv.org/abs/2603.15644)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.24102#S2.SS3.p1.1),[§2\.4\.3](https://arxiv.org/html/2606.24102#S2.SS4.SSS3.p1.1),[§2\.4\.4](https://arxiv.org/html/2606.24102#S2.SS4.SSS4.p1.3)\.
- \[12\]L\. L\. Guo, S\. E\. Arciniegas, A\. P\. Yan, J\. Fries, G\. A\. Tomlinson, and L\. Sung\(2026\)Systematic review of foundation models for structured electronic health records\.Journal of the American Medical Informatics Association33\(6\),pp\. 1190–1198\.External Links:ISSN 1527\-974X,[Document](https://dx.doi.org/10.1093/jamia/ocag033),[Link](https://doi.org/10.1093/jamia/ocag033)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p2.1)\.
- \[13\]L\. L\. Guo, M\. Calligan, E\. Vettese, S\. Cook, G\. Gagnidze, O\. Han, J\. Inoue, J\. Lemmon, J\. Li, M\. Roshdi, B\. Sadovy, S\. Wallace, and L\. Sung\(2023\)Development and validation of the SickKids enterprise\-wide data in Azure repository \(SEDAR\)\.Heliyon9\(11\),pp\. e21586\.External Links:ISSN 2405\-8440,[Document](https://dx.doi.org/10.1016/j.heliyon.2023.e21586),[Link](https://www.sciencedirect.com/science/article/pii/S2405844023087947)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p2.1)\.
- \[14\]L\. L\. Guo, J\. Fries, E\. Steinberg, S\. L\. Fleming, K\. Morse, C\. Aftandilian, J\. Posada, N\. Shah, and L\. Sung\(2024\)A multi\-center study on the adaptability of a shared foundation model for electronic health records\.npj Digital Medicine7\(1\),pp\. 171\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-024-01166-w),[Link](https://doi.org/10.1038/s41746-024-01166-w)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[15\]L\. L\. Guo, E\. Steinberg, S\. L\. Fleming, J\. Posada, J\. Lemmon, S\. R\. Pfohl, N\. Shah, J\. Fries, and L\. Sung\(2023\)EHR foundation models improve robustness in the presence of temporal distribution shift\.Scientific Reports13\(1\),pp\. 3767\.External Links:ISSN 2045\-2322,[Document](https://dx.doi.org/10.1038/s41598-023-30820-8),[Link](https://doi.org/10.1038/s41598-023-30820-8)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[16\]S\. Gururangan, A\. Marasovic, S\. Swayamdipta, K\. Lo, I\. Beltagy, D\. Downey, and N\. A\. Smith\(2020\)Don’t stop pretraining: adapt language models to domains and tasks\.Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 8342–8360\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.740),[Link](https://aclanthology.org/2020.acl-main.740/)Cited by:[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[17\]S\. Hegselmann, G\. v\. Arnim, T\. Rheude, N\. Kronenberg, D\. A\. Sontag, G\. Hindricks, R\. Eils, and B\. Wild\(2025\)Large language models are powerful electronic health record encoders\.arXiv2502\.17403\.External Links:[Link](https://arxiv.org/abs/2502.17403)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§2\.5](https://arxiv.org/html/2606.24102#S2.SS5.p3.1),[§4](https://arxiv.org/html/2606.24102#S4.p5.1)\.
- \[18\]S\. Holm\(1979\)A simple sequentially rejective multiple test procedure\.Scandinavian Journal of Statistics6,pp\. 65–70\.Cited by:[§2\.10](https://arxiv.org/html/2606.24102#S2.SS10.p1.1)\.
- \[19\]K\. Hur, H\. Kwak, J\. Jang, N\. Kim, and E\. Choi\(2026\)Multi\-lingual multi\-institutional electronic health record based predictive model\.arXiv2604\.00027\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2604.00027),[Link](https://arxiv.org/abs/2604.00027)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§4](https://arxiv.org/html/2606.24102#S4.p2.1),[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[20\]K\. Hur, J\. Lee, J\. Oh, W\. Price, Y\. Kim, and E\. Choi\(2022\)Unifying heterogeneous electronic health records systems via text\-based code embedding\.Conference ProceedingsInProceedings of the Conference on Health, Inference, and Learning,Proceedings of Machine Learning Research, Vol\.174,pp\. 183–203\.External Links:[Link](https://proceedings.mlr.press/v174/hur22a.html)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§4](https://arxiv.org/html/2606.24102#S4.p2.1)\.
- \[21\]K\. Hur, J\. Oh, J\. Kim, J\. Kim, M\. J\. Lee, E\. Cho, S\. Moon, Y\. Kim, L\. Atallah, and E\. Choi\(2024\)GenHPF: general healthcare predictive framework for multi\-task multi\-source learning\.IEEE Journal of Biomedical and Health Informatics28\(1\),pp\. 502–513\.External Links:[Document](https://dx.doi.org/10.1109/JBHI.2023.3327951),[Link](https://doi.org/10.1109/JBHI.2023.3327951)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§4](https://arxiv.org/html/2606.24102#S4.p2.1)\.
- \[22\]Z\. Jing, V\. Jeanselme, Y\. Kobayashi, S\. A\. Lee, C\. Pang, A\. Kashyap, Y\. Li, X\. Jiang, and S\. Joshi\(2026\)One loss to rule them all: marked time\-to\-event for structured EHR foundation models\.arXiv2602\.00541\.External Links:[Link](https://arxiv.org/abs/2602.00541)Cited by:[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[23\]A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. Mark\(2023\)MIMIC\-IV, a freely accessible electronic health record dataset\.Sci Data10\(1\),pp\. 1\.External Links:ISSN 2052\-4463,[Document](https://dx.doi.org/10.1038/s41597-022-01899-x)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p2.1)\.
- \[24\]J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei\(2020\)Scaling laws for neural language models\.arXiv2001\.08361\.External Links:[Link](https://arxiv.org/abs/2001.08361)Cited by:[§2\.9](https://arxiv.org/html/2606.24102#S2.SS9.p1.1)\.
- \[25\]M\. Kirchler, M\. Ferro, V\. Lorenzini, R\. P\. van de Water, C\. Lippert, A\. Ganna, and FinnGen\(2026\)Large language models improve transferability of electronic health record\-based predictions across countries and coding systems\.npj Digital Medicine9\(1\),pp\. 177\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-026-02363-5),[Link](https://doi.org/10.1038/s41746-026-02363-5)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1),[§4](https://arxiv.org/html/2606.24102#S4.p2.1)\.
- \[26\]J\. Lemmon, L\. L\. Guo, E\. Steinberg, K\. E\. Morse, S\. L\. Fleming, C\. Aftandilian, S\. R\. Pfohl, J\. D\. Posada, N\. Shah, J\. Fries, and L\. Sung\(2023\)Self\-supervised machine learning using adult inpatient data produces effective models for pediatric clinical prediction tasks\.Journal of the American Medical Informatics Association30\(12\),pp\. 2004–2011\.External Links:ISSN 1527\-974X,[Document](https://dx.doi.org/10.1093/jamia/ocad175),[Link](https://doi.org/10.1093/jamia/ocad175)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[27\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.International Conference on Learning Representations\.External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§2\.4\.4](https://arxiv.org/html/2606.24102#S2.SS4.SSS4.p1.3)\.
- \[28\]N\. Muennighoff, A\. M\. Rush, B\. Barak, T\. Le Scao, A\. Piktus, N\. Tazi, S\. Pyysalo, T\. Wolf, and C\. Raffel\(2023\)Scaling data\-constrained language models\.Advances in Neural Information Processing Systems36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/9d89448b63ce1e2e8dc7af72c984c196-Abstract-Conference.html)Cited by:[§2\.4\.4](https://arxiv.org/html/2606.24102#S2.SS4.SSS4.p1.3)\.
- \[29\]N\. Muennighoff, N\. Tazi, L\. Magne, and N\. Reimers\(2023\)MTEB: massive text embedding benchmark\.Conference ProceedingsInConference of the European Chapter of the Association for Computational Linguistics,pp\. 2014–2037\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.eacl-main.148),[Link](https://aclanthology.org/2023.eacl-main.148/)Cited by:[§4](https://arxiv.org/html/2606.24102#S4.p4.1)\.
- \[30\]OHDSI\(2024\)MIMIC \(medical information mart for intensive care\) \- OMOP CDM ETL repository\.Web Page\.External Links:[Link](https://github.com/OHDSI/MIMIC)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p2.1)\.
- \[31\]C\. Pang, J\. Park, X\. Jiang, N\. P\. Pavinkurve, K\. S\. Kalluri, S\. Joshi, N\. Elhadad, and K\. Natarajan\(2025\)CEHR\-XGPT: a scalable multi\-task foundation model for electronic health records\.arXiv2509\.03643\.External Links:[Link](https://arxiv.org/abs/2509.03643)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[32\]F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg, J\. Vanderplas, A\. Passos, D\. Cournapeau, M\. Brucher, M\. Perrot, and É\. Duchesnay\(2011\)Scikit\-learn: machine learning in Python\.Journal of Machine Learning Research12,pp\. 2825–2830\.External Links:[Link](https://jmlr.org/papers/v12/pedregosa11a.html)Cited by:[§2\.7\.1](https://arxiv.org/html/2606.24102#S2.SS7.SSS1.p1.1)\.
- \[33\]E\. Perez, F\. Strub, H\. de Vries, V\. Dumoulin, and A\. Courville\(2018\)FiLM: visual reasoning with a general conditioning layer\.Proceedings of the AAAI Conference on Artificial Intelligence32\(1\)\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v32i1.11671),[Link](https://doi.org/10.1609/aaai.v32i1.11671)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p5.1)\.
- \[34\]L\. Rasmy, Y\. Xiang, Z\. Xie, C\. Tao, and D\. Zhi\(2021\)Med\-BERT: pretrained contextualized embeddings on large\-scale structured electronic health records for disease prediction\.npj Digital Medicine4\(1\),pp\. 86\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-021-00455-y),[Link](https://doi.org/10.1038/s41746-021-00455-y)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[35\]F\. Remy, K\. Demuynck, and T\. Demeester\(2024\)BioLORD\-2023: semantic textual representations fusing large language models and clinical knowledge graph insights\.Journal of the American Medical Informatics Association31\(9\),pp\. 1844–1855\.External Links:[Document](https://dx.doi.org/10.1093/jamia/ocae029),[Link](https://doi.org/10.1093/jamia/ocae029)Cited by:[§2\.4\.1](https://arxiv.org/html/2606.24102#S2.SS4.SSS1.p1.1)\.
- \[36\]P\. Renc, Y\. Jia, A\. E\. Samir, J\. Was, Q\. Li, D\. W\. Bates, and A\. Sitek\(2024\)Zero shot health trajectory prediction using transformer\.npj Digital Medicine7\(1\),pp\. 256\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-024-01235-0),[Link](https://doi.org/10.1038/s41746-024-01235-0)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1),[§2\.5](https://arxiv.org/html/2606.24102#S2.SS5.p1.1)\.
- \[37\]A\. Shmatko, A\. W\. Jung, K\. Gaurav, S\. Brunak, L\. H\. Mortensen, E\. Birney, T\. Fitzgerald, and M\. Gerstung\(2025\)Learning the natural history of human disease with generative transformers\.Nature647\(8088\),pp\. 248–256\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/s41586-025-09529-3),[Link](https://doi.org/10.1038/s41586-025-09529-3)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1),[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[38\]E\. Steinberg, J\. Fries, Y\. Xu, and N\. Shah\(2024\)MOTOR: a time\-to\-event foundation model for structured medical records\.International Conference on Learning Representations\.External Links:[Link](https://openreview.net/forum?id=NialiwI2V6)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1),[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[39\]E\. Steinberg, K\. Jung, J\. A\. Fries, C\. K\. Corbin, S\. R\. Pfohl, and N\. H\. Shah\(2021\)Language models are an effective representation learning technique for electronic health record data\.Journal of Biomedical Informatics113,pp\. 103637\.External Links:ISSN 1532\-0464,[Document](https://dx.doi.org/10.1016/j.jbi.2020.103637),[Link](https://www.sciencedirect.com/science/article/pii/S1532046420302653)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1),[§2\.5](https://arxiv.org/html/2606.24102#S2.SS5.p1.1)\.
- \[40\]E\. Steinberg, M\. Wornow, S\. Bedi, J\. A\. Fries, M\. B\. A\. McDermott, and N\. H\. Shah\(2024\)Meds\_reader: a fast and efficient EHR processing library\.arXiv2409\.09095\.External Links:[Link](https://arxiv.org/abs/2409.09095)Cited by:[§2\.1](https://arxiv.org/html/2606.24102#S2.SS1.p2.1)\.
- \[41\]J\. Su, Y\. Lu, S\. Pan, A\. Murtadha, B\. Wen, and Y\. Liu\(2021\)RoFormer: enhanced transformer with rotary position embedding\.arXiv2104\.09864\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2104.09864),[Link](https://arxiv.org/abs/2104.09864)Cited by:[§2\.4\.3](https://arxiv.org/html/2606.24102#S2.SS4.SSS3.p1.1)\.
- \[42\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.Advances in Neural Information Processing Systems30\.External Links:[Link](https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html)Cited by:[§2\.4\.3](https://arxiv.org/html/2606.24102#S2.SS4.SSS3.p1.1)\.
- \[43\]P\. Virtanen, R\. Gommers, T\. E\. Oliphant, M\. Haberland, T\. Reddy, D\. Cournapeau, E\. Burovski, P\. Peterson, W\. Weckesser, J\. Bright, S\. J\. van der Walt, M\. Brett, J\. Wilson, K\. J\. Millman, N\. Mayorov, A\. R\. J\. Nelson, E\. Jones, R\. Kern, E\. Larson, C\. J\. Carey, İ\. Polat, Y\. Feng, E\. W\. Moore, J\. VanderPlas, D\. Laxalde, J\. Perktold, R\. Cimrman, I\. Henriksen, E\. A\. Quintero, C\. R\. Harris, A\. M\. Archibald, A\. H\. Ribeiro, F\. Pedregosa, P\. van Mulbregt, A\. Vijaykumar, A\. P\. Bardelli, A\. Rothberg, A\. Hilboll, A\. Kloeckner, A\. Scopatz, A\. Lee, A\. Rokem, C\. N\. Woods, C\. Fulton, C\. Masson, C\. Häggström, C\. Fitzgerald, D\. A\. Nicholson, D\. R\. Hagen, D\. V\. Pasechnik, E\. Olivetti, E\. Martin, E\. Wieser, F\. Silva, F\. Lenders, F\. Wilhelm, G\. Young, G\. A\. Price, G\. Ingold, G\. E\. Allen, G\. R\. Lee, H\. Audren, I\. Probst, J\. P\. Dietrich, J\. Silterra, J\. T\. Webber, J\. Slavič, J\. Nothman, J\. Buchner, J\. Kulick, J\. L\. Schönberger, J\. V\. de Miranda Cardoso, J\. Reimer, J\. Harrington, J\. L\. C\. Rodríguez, J\. Nunez\-Iglesias, J\. Kuczynski, K\. Tritz, M\. Thoma, M\. Newville, M\. Kümmerer, M\. Bolingbroke, M\. Tartre, M\. Pak, N\. J\. Smith, N\. Nowaczyk, N\. Shebanov, O\. Pavlyk, P\. A\. Brodtkorb, P\. Lee, R\. T\. McGibbon, R\. Feldbauer, S\. Lewis, S\. Tygier, S\. Sievert, S\. Vigna, S\. Peterson, S\. More, T\. Pudlik, T\. Oshima,et al\.\(2020\)SciPy 1\.0: fundamental algorithms for scientific computing in Python\.Nature Methods17\(3\),pp\. 261–272\.External Links:ISSN 1548\-7105,[Document](https://dx.doi.org/10.1038/s41592-019-0686-2),[Link](https://doi.org/10.1038/s41592-019-0686-2)Cited by:[§2\.10](https://arxiv.org/html/2606.24102#S2.SS10.p1.1)\.
- \[44\]S\. Waxler, P\. Blazek, D\. White, D\. Sneider, K\. Chung, M\. Nagarathnam, P\. Williams, H\. Voeller, K\. Wong, M\. Swanhorst, S\. Zhang, N\. Usuyama, C\. Wong, T\. Naumann, H\. Poon, A\. Loza, D\. Meeker, S\. Hain, and R\. Shah\(2025\)Generative medical event models improve with scale\.arXiv2508\.12104\.External Links:[Link](https://arxiv.org/abs/2508.12104)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p1.1)\.
- \[45\]F\. Wilcoxon\(1945\)Individual comparisons by ranking methods\.Biometrics Bulletin1\(6\),pp\. 80–83\.External Links:[Document](https://dx.doi.org/10.2307/3001968)Cited by:[§2\.10](https://arxiv.org/html/2606.24102#S2.SS10.p1.1)\.
- \[46\]L\. Xu, H\. Xie, S\. J\. Qin, X\. Tao, and F\. L\. Wang\(2026\)Parameter\-efficient fine\-tuning methods for pretrained language models: a critical review and assessment\.IEEE Transactions on Pattern Analysis and Machine Intelligence48\(6\),pp\. 6107–6126\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2026.3657354),[Link](https://doi.org/10.1109/TPAMI.2026.3657354)Cited by:[§4](https://arxiv.org/html/2606.24102#S4.p6.1)\.
- \[47\]Y\. Zhang, M\. Li, D\. Long, X\. Zhang, H\. Lin, B\. Yang, P\. Xie, A\. Yang, D\. Liu, J\. Lin, F\. Huang, and J\. Zhou\(2025\)Qwen3 embedding: advancing text embedding and reranking through foundation models\.arXiv2506\.05176\.External Links:[Link](https://arxiv.org/abs/2506.05176)Cited by:[§2\.4\.1](https://arxiv.org/html/2606.24102#S2.SS4.SSS1.p1.1)\.
- \[48\]D\. Zhou, H\. Tong, L\. Wang, S\. Liu, X\. Xiong, Z\. Gan, R\. Griffier, B\. P\. Hejblum, Y\. Liu, C\. Hong, C\. Bonzel, T\. Cai, K\. Pan, Y\. Ho, L\. Costa, V\. A\. Panickan, J\. M\. Gaziano, K\. D\. Mandl, V\. Jouhet, R\. Thiebaut, Z\. Xia, K\. Cho, K\. Liao, and T\. Cai\(2026\)Representation learning to advance multi\-institutional studies with electronic health record data from us and france\.Nature Communications17\(1\),pp\. 4800\.External Links:ISSN 2041\-1723,[Document](https://dx.doi.org/10.1038/s41467-026-71152-1),[Link](https://doi.org/10.1038/s41467-026-71152-1)Cited by:[§1](https://arxiv.org/html/2606.24102#S1.p4.1)\.

## Supplementary Material

### Supplementary Table S1\. Comparison of PORTER with related text\-based approaches for structured EHR modeling

ApproachMethodological featuresEvaluationText representation granularityaText encoderbEHR pretraining objectiveFrozen patient representation reused across tasksExplicit numeric value handlingCross\-siteCross\-vocabulary \(same patient\)cHegselmann et al\. \[15\]Patient\-levelFrozen \(pretrained\)——Partial✓—LRRL \[14\]Patient\-levelFrozen \(pretrained\)——Partial——DescEmb \[19\]Code\- or Event\-levelTrain or fine\-tune——Partial✓—GenHPF \[22\]Code\- or Event\-levelTrain or fine\-tuneMasked \+ contrastive—Partial✓—Hur et al\. 2026 \[23\]Code\- or Event\-levelTrain or fine\-tune——Partial✓—GRASP \[18\]Code\- or Event\-levelFrozen \(pretrained\)———✓—GAME \[20\]Code\- or Event\-levelFrozen \(pretrained\)———✓—Attrach et al\. \[21\]Code\- or Event\-levelFrozen \(pretrained\)——✓——PORTERCode\- or Event\-levelFrozen \(pretrained\)Autoregressive \(next\-event prediction\)✓✓✓✓Symbols\. ✓indicates present; — indicates absent or not applicable\.

aPatient\-level serializes the whole record into a single text document\. Code\- or event\-level renders each code or clinical event as text\. For GAME, text is one of several inputs to its code embeddings\.

bReflects each method's primary or representative encoder configuration\. Frozen \(pretrained\): a pretrained text encoder used without weight updates; train or fine\-tune: encoder weights updated, from random or pretrained initialization\.

cThe cross\-vocabulary evaluation isolates vocabulary shift by holding patients, labels, and clinical events fixed and changing only the naming system

Abbreviations: EHR, electronic health record; FM, foundation model; LRRL, large language model rubric representation learning; GRASP, Generalizable Risk Assessment with Semantic Projection; GAME, graph alignment for multi\-institutional EHR data; PORTER, Portable EHR Representations\.

### Supplementary Figure S1\. Cohort construction for pretraining and downstream evaluation

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x3.png)

Pretraining \(left\) used a patient\-level random split, with different temporal cutoffs for training and validation to evaluate models on more recent clinical data during pretraining\. Downstream evaluation \(right\) used admission\-level temporal splits across 74 clinical prediction tasks, aligned to the same calendar cutoffs as pretraining\. The unit of analysis was patients for pretraining and admissions for downstream prediction\.

Abbreviations: EHR, electronic health records; SickKids, The Hospital for Sick Children\.

### Supplementary Table S2\. SickKids and MIMIC task cohort statistics

TaskTotal admissionsTotal patientsPositive casesPrevalence \(%\)SickKidsTransfusionsPlatelet transfusion97,63956,8603,3263\.4Red cell transfusion92,09054,1007,3678\.0ProcedureInvasive intubation95,68555,5691,7821\.9Gastrostomy tube99,20957,3824700\.5Echocardiogram97,16856,3737,7588\.0Pulmonary function test99,22357,3838480\.9Lumbar puncture98,24157,0182,5462\.6Surgery76,75743,69813,62317\.7Interventional radiology98,25157,1798,3098\.5ImagingPlain radiography chest88,43152,13310,41111\.8Ultrasound abdomen96,51356,32511,98112\.4Computerized tomography chest98,88057,2732,3522\.4Computerized tomography abdomen98,86857,2581,4281\.4Computerized tomography head97,79956,7433,3983\.5MRI head97,35056,4956,0156\.2MRI whole body99,24957,3821810\.2PET99,25457,3862230\.2Laboratory abnormalityHigh white blood count90,11153,05512,82914\.2Low white blood count95,16555,9809,83510\.3High absolute neutrophil count91,48553,61711,34812\.4Low absolute neutrophil count97,79156,9277,3937\.6High bands93,61554,67013,28814\.2High lymphocyte97,52556,6944,6644\.8Low lymphocyte92,86754,77313,94815\.0High hemoglobin95,40955,7404,3354\.5Low hemoglobin86,50351,94619,57122\.6High mean corpuscular volume95,70756,2926,7337\.0Low mean corpuscular volume95,57655,8045,0665\.3High reticulocyte count98,11457,0374,2564\.3Low reticulocyte count98,55057,0493,2573\.3High platelet94,09955,39812,40213\.2Low platelet90,82353,94913,11814\.4High immature platelet fraction97,24556,6845,8046\.0Low immature platelet fraction98,51257,0472,7442\.8High mean platelet volume97,64756,8024,9045\.0Low mean platelet volume94,81955,3178,6009\.1High fibrinogen98,62657,0332,3572\.4Low fibrinogen96,22055,7531,8211\.9High partial thromboplastin time96,05055,9893,7343\.9High international normalized ratio92,24354,1457,3227\.9High sodium91,13453,50810,15811\.1Low sodium95,31955,8798,5298\.9High potassium94,30455,2288,1038\.6Low potassium90,25353,17414,36715\.9High glucose86,73951,19712,28014\.2Low glucose97,17656,6994,1744\.3High creatinine95,55755,9565,3275\.6High urea97,73556,9673,5793\.7Low albumin95,51455,84011,95012\.5High alanine transaminase94,75055,7099,2359\.7High aspartate aminotransferase95,70656,0087,7278\.1High lactate dehydrogenase98,30656,9782,6532\.7High bilirubin94,63755,2605,6826\.0High cholesterol99,09157,3305960\.6High triglyceride98,59357,0262,5272\.6High ferritin98,06656,7863,5423\.6High creatinine kinase98,76557,0898590\.9High C\-reactive protein92,87054,44712,66613\.6High erythrocyte sedimentation rate98,18956,8723,0513\.1Low PaO298,39456,9702,8292\.9Low SpO267,90140,39823,55634\.7MedicationsAny antibacterial62,44839,29019,76831\.7Any antifungal98,45257,3312,4242\.5Any chemotherapy94,60457,1843,4603\.7Any antiepileptics88,67254,1127,5458\.5Any glucocorticoid76,12545,35015,97221\.0Dexamethasone79,73746,57711,14714\.0Any opioid67,83039,18418,10926\.7Morphine76,86944,40313,73317\.9Fentanyl79,10145,63613,95917\.6Any inotrope95,06255,1802,1542\.3Clinical outcomesLong length of stay \(≥\\geq7 days\)99,36657,43722,04622\.2Readmission within 30 days100,14957,76717,16817\.1Mortality99,25857,3875800\.6MIMICLaboratory abnormalityHigh white blood count44,61334,69612,03627\.0Low white blood count57,09143,3234,0907\.2High absolute neutrophil count54,51941,1653,4706\.4Low absolute neutrophil count58,32143,9615881\.0High lymphocyte58,31143,9243270\.6Low lymphocyte55,00241,8882,6844\.9Low hemoglobin35,49928,99215,39743\.4High mean corpuscular volume54,44741,3924,2737\.8Low mean corpuscular volume54,40341,5872,3924\.4High platelet56,92443,1793,1695\.6Low platelet52,21839,8928,88917\.0High fibrinogen57,32143,3512,2053\.8Low fibrinogen57,96743,6171,3042\.2High partial thromboplastin time52,68040,0398,12815\.4High international normalized ratio41,93032,14510,43024\.9High sodium57,39043,2413,9006\.8Low sodium54,75941,5268,45715\.4High potassium55,91542,3486,16811\.0Low potassium56,57042,6788,11814\.4High glucose39,08430,20414,42836\.9Low glucose57,96743,7153,7326\.4High creatinine48,97337,5566,81513\.9High urea46,28635,6099,24120\.0Low albumin53,53940,6027,74814\.5High alanine transaminase54,01740,9064,7528\.8High aspartate aminotransferase53,12540,1615,2469\.9High lactate dehydrogenase55,09441,7494,9749\.0High bilirubin56,06642,2122,6114\.7High cholesterol58,30743,8954750\.8High triglyceride58,08343,7361,6322\.8High ferritin57,90943,6152,2673\.9High creatinine kinase57,03742,8831,5082\.6High C\-reactive protein57,91143,6671,7283\.0Clinical outcomesLong length of stay \(≥\\geq7 days\)58,51344,05517,21829\.4Readmission within 30 days58,51244,0553,1435\.4Mortality58,51344,0551,7413\.0Abbreviations: MIMIC, Medical Information Mart for Intensive Care; SickKids, The Hospital for Sick Children\.

### Supplementary Table S3\. OMOP\-derived event description templates and source columns

CategoryTemplate segmentsSlot mapping to OMOP\-derived fieldsbdemographicsPatient Birth DateaNonedemographicsPatient Death DateaNonedemographicsPatient Sex: \{sex\}sex: gender\_concept\_nameMeasurement\*Measurement: \{name\}Result: \{result\}name: measurement\_concept\_nameresult: meas\_value\_concept\_nameConditionCondition: \{name\}name: condition\_concept\_nameObservationObservation: \{name\}name: observation\_concept\_nameProcedureProcedure: \{name\}name: procedure\_concept\_nameDrug\*Drug: \{name\}Route: \{route\}name: drug\_concept\_nameroute: drug\_route\_concept\_namenoteNote: \{name\}name: note\_concept\_namespecimenSpecimen: \{name\}name: spec\_anatomic\_site\_concept\_namevisitVisit: \{name\}name: visit\_concept\_name\*Template segments whose source columns are NULL are dropped\. For example, measurement events emit "Measurement: \{name\}\. Result: \{result\}" when meas\_value\_concept\_name is not null, and only "Measurement: \{name\}" when meas\_value\_concept\_name is null\.

aPatient birth date and patient death date were represented as fixed demographic event descriptions\.

bFields ending in \_concept\_name are derived fields obtained by joining the corresponding OMOP concept identifier field, such as measurement\_concept\_id, condition\_concept\_id, or drug\_concept\_id, to the OMOP concept table and using the associated concept\_name\. Analogous SEDAR\-derived concept and attribute fields were used to construct event descriptions for cross\-vocabulary evaluation\.

Abbreviations: OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise\-wide Data in Azure Repository\.

### Supplementary Figure S2\. Pretraining loss curves for PORTER using different text encoders

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x4.png)

Cross\-entropy loss during pretraining for the four text encoder variants\. Training loss was logged every 10 steps\. Held\-out evaluation loss was logged every 250 steps\. Faded traces show raw values\. Bold traces show exponential moving averages \(span 50 for training, 10 for evaluation\)\. Upper\-right insets magnify the last ~7,000 steps so late\-training differences are legible\.

### Supplementary Figure S3\. Empirical distribution of pre\-clip numeric scalar features

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x5.png)

Distributions of the log1p fallback scalar \(left\) and reference\-range position scalar \(right\) are shown before clipping in a sample \(n=50,000\) from the SickKids training split\.

### Supplementary Table S4\. Training, model, and evaluation hyperparameters

HyperparameterValuePretraining optimization and batchingOptimizerAdamWLearning rate5e\-4Learning rate schedulercosine\_with\_min\_lrNum epochs5Gradient accumulation steps2Early stoppingNone\*Max clinical events per batch16,384\*Min patients per batch1Weight decay0\.05Max gradient norm \(clipping\)1\.0Warmup steps150Adamβ\\beta10\.9Adamβ\\beta20\.95Floating\-point formatbf16Transformer backboneHidden size768Num layers28Num attention heads12Intermediate size768ActivationSwiGLU\*\*Alternating dense layersYes\*\*Attention width128Linear probe \(Logistic Regression\)Input preprocessingStandardScalerSolverLBFGSRegularizationL2Inverse regularization \(C\)1, 0\.1, 0\.01, 0\.001, 0\.0001Max iterations10,000\*Batches were constructed using a fixed clinical\-event budget \(16,384 clinical events per batch\) with a minimum of one patient per batch to accommodate variable\-length patient sequences without padding\. This imposes an effective upper bound of 16,384 events per patient sequence \(i\.e\., maximum context window\)\. When multiple patient sequences were in a batch, causal masking prevented attention across patient boundaries, ensuring independent sequence modeling\.

\*\* The transformer alternates between global and local self\-attention, starting with a global attention layer, followed by a local attention layer \(attention width of 128\), and repeating this pattern throughout\.

### Supplementary Figure S4\. Illustrative serialized patient timeline for the text serialization comparator

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x6.png)

Text serialization of a single synthetic patient's event stream into Markdown for the text serialization comparator\. The full document is prefixed with the task\-specific instruction \(Supplementary Table S4\) before encoding with Qwen3\-Embedding\-8B\.

Abbreviation: LLM, large language model\.

### Supplementary Table S5\. Task\-specific retrieval instructions used by the text serialization comparator

TaskQuery \(verbatim\)TransfusionsPlatelet transfusionwill the patient need a platelet transfusionRed cell transfusionwill the patient need a red blood cell transfusionProcedureInvasive intubationwill the patient require invasive intubationGastrostomy tubewill the patient need a gastrostomy tube placementEchocardiogramwill the patient need an echocardiogramPulmonary function testwill the patient need a pulmonary function testLumbar puncturewill the patient need a lumbar punctureSurgerywill the patient need surgeryInterventional radiologywill the patient need an image\-guided therapy procedureImagingPlain radiography chestwill the patient need a chest x\-rayUltrasound abdomenwill the patient need an abdominal ultrasoundComputerized tomography chestwill the patient need a chest CT scanComputerized tomography abdomenwill the patient need an abdominal CT scanComputerized tomography headwill the patient need a head CT scanMRI headwill the patient need a head MRIMRI whole bodywill the patient need a whole body MRIPETwill the patient need a PET scanLaboratory abnormalityHigh white blood countwill the patient have a high white blood cell countLow white blood countwill the patient have a low white blood cell countHigh absolute neutrophil countwill the patient have a high absolute neutrophil countLow absolute neutrophil countwill the patient have a low absolute neutrophil countHigh bandswill the patient have a high band neutrophil countHigh lymphocytewill the patient have a high lymphocyte countLow lymphocytewill the patient have a low lymphocyte countHigh hemoglobinwill the patient have a high hemoglobin levelLow hemoglobinwill the patient have a low hemoglobin levelHigh mean corpuscular volumewill the patient have a high mean corpuscular volumeLow mean corpuscular volumewill the patient have a low mean corpuscular volumeHigh reticulocyte countwill the patient have a high absolute reticulocyte countLow reticulocyte countwill the patient have a low absolute reticulocyte countHigh plateletwill the patient have a high platelet countLow plateletwill the patient have a low platelet countHigh immature platelet fractionwill the patient have a high immature platelet fractionLow immature platelet fractionwill the patient have a low immature platelet fractionHigh mean platelet volumewill the patient have a high mean platelet volumeLow mean platelet volumewill the patient have a low mean platelet volumeHigh fibrinogenwill the patient have a high fibrinogen levelLow fibrinogenwill the patient have a low fibrinogen levelHigh partial thromboplastin timewill the patient have a high partial thromboplastin timeHigh international normalized ratiowill the patient have a high international normalized ratioHigh sodiumwill the patient have a high sodium levelLow sodiumwill the patient have a low sodium levelHigh potassiumwill the patient have a high potassium levelLow potassiumwill the patient have a low potassium levelHigh glucosewill the patient have a high blood glucose levelLow glucosewill the patient have a low blood glucose levelHigh creatininewill the patient have a high creatinine levelHigh ureawill the patient have a high urea levelLow albuminwill the patient have a low albumin levelHigh alanine transaminasewill the patient have a high alanine aminotransferase levelHigh aspartate aminotransferasewill the patient have a high aspartate aminotransferase levelHigh lactate dehydrogenasewill the patient have a high lactate dehydrogenase levelHigh bilirubinwill the patient have a high bilirubin levelHigh cholesterolwill the patient have a high cholesterol levelHigh triglyceridewill the patient have a high triglyceride levelHigh ferritinwill the patient have a high ferritin levelHigh creatinine kinasewill the patient have a high creatine kinase levelHigh C\-reactive proteinwill the patient have a high C\-reactive protein levelHigh erythrocyte sedimentation ratewill the patient have a high erythrocyte sedimentation rateLow PaO2will the patient have a low partial pressure of oxygenLow SpO2will the patient have low oxygen saturationMedicationsAny antibacterialwill the patient need antibacterial medicationAny antifungalwill the patient need antifungal medicationAny chemotherapywill the patient need chemotherapyAny antiepilepticswill the patient need antiepileptic medicationAny glucocorticoidwill the patient need glucocorticoid medicationDexamethasonewill the patient need dexamethasoneAny opioidwill the patient need opioid medicationMorphinewill the patient need morphineFentanylwill the patient need fentanylAny inotropewill the patient need inotrope or vasopressor medicationClinical outcomesLong length of stay \(≥\\geq7 days\)will the patient stay in the hospital for more than 7 daysReadmission within 30 dayswill the patient be readmitted to the hospital within 30 daysMortalitywill the patient die during this hospital admissionAbbreviations: LLM, large language model\.

### Supplementary Table S6\. Pretraining compute and parameter size by model

Model\*Frozen lookup paramsTrainable input paramsTotal trainable paramsTotal FLOPsPORTER15\.12 M1\.64 M192\.87 M1\.01×\\times1018Fixed\-Vocab FM075\.50 M266\.72 M9\.98×\\times1017PORTER\-Qwen380\.63 M4\.20 M195\.42 M1\.02×\\times1018PORTER\-BGE\-M320\.16 M1\.84 M193\.06 M1\.01×\\times1018PORTER\-Random80\.63 M4\.20 M195\.42 M1\.02×\\times1018PORTER\-NoNum15\.12 M1\.18 M192\.41 M1\.00×\\times1018PORTER\-NumText876\.74 M1\.18 M192\.41 M1\.00×\\times1018\* The transformer backbone \(115\.63 M\), next\-event prediction head \(75\.60 M\), and number of events seen \(894 M\) are fixed across all models\. All parameter counts are reported in millions \(M\)\. Differences in trainable parameters arise from the input pathway: 1\) the learnable joint embedding table \(i\.e\., Fixed\-Vocab FM, 75\.50 M\); 2\) the projection MLP from each text encoder's native dimension to the backbone hidden dimension; 3\) the FiLM numeric pathway present in all PORTER variants except PORTER\-NoNum and PORTER\-NumText, which render numeric values as text or omit them entirely\.

Abbreviations: FiLM, feature\-wise linear modulation; FLOPs, floating\-point operations; FM, foundation model; M, million; MLP, multilayer perceptron\.

### Supplementary Table S7\. Per\-task AUROC for PORTER and Fixed\-Vocab FM across in\-domain, cross\-vocabulary, and cross\-site settings

TaskIn\-domain \(SK\)Cross\-vocabulary \(SK\)Cross\-site \(MIMIC\)PORTERFixed\-Vocab FMPORTERFixed\-Vocab FMaTarget\-vocabulary
Fixed\-Vocab FM referencebPORTERFixed\-Vocab FMTarget\-site
Fixed\-Vocab FM referencecTransfusionsPlatelet transfusion0\.9670\.9700\.944–0\.961–––Red cell transfusion0\.9290\.9260\.891–0\.919–––ProcedureInvasive intubation0\.9240\.9250\.894–0\.931–––Gastrostomy tube0\.8850\.8860\.855–0\.903–––Echocardiogram0\.8940\.8920\.869–0\.890–––Pulmonary function test0\.9660\.9660\.941–0\.962–––Lumbar puncture0\.9470\.9410\.916–0\.942–––Surgery0\.8910\.8870\.850–0\.894–––Interventional radiology0\.8640\.8640\.815–0\.860–––ImagingPlain radiography chest0\.8200\.8200\.787–0\.824–––Ultrasound abdomen0\.8410\.8470\.810–0\.836–––Computerized tomography chest0\.8680\.8660\.818–0\.852–––Computerized tomography abdomen0\.8940\.8910\.824–0\.883–––Computerized tomography head0\.9100\.9090\.869–0\.911–––MRI head0\.9080\.9120\.875–0\.912–––MRI whole body0\.9480\.9440\.903–0\.913–––PET0\.8970\.9010\.823–0\.801–––Laboratory abnormalityHigh white blood count0\.8340\.8300\.781–0\.7950\.7600\.7510\.803Low white blood count0\.9200\.9160\.883–0\.8950\.8690\.8760\.905High absolute neutrophil count0\.8390\.8400\.788–0\.8140\.7950\.7610\.818Low absolute neutrophil count0\.9340\.9340\.906–0\.9170\.8530\.8520\.886High bands0\.8720\.8740\.845–0\.868–––High lymphocyte0\.8520\.8480\.816–0\.8300\.7130\.6800\.737Low lymphocyte0\.8940\.8950\.871–0\.8870\.8160\.7930\.831High hemoglobin0\.8610\.8480\.818–0\.839–––Low hemoglobin0\.8870\.8820\.855–0\.8750\.8590\.8500\.876High mean corpuscular volume0\.8920\.8950\.860–0\.8850\.7930\.8390\.813Low mean corpuscular volume0\.8560\.8380\.779–0\.8040\.8060\.8560\.766High reticulocyte count0\.8980\.8940\.855–0\.884–––Low reticulocyte count0\.8930\.8870\.851–0\.851–––High platelet0\.8410\.8390\.805–0\.8270\.7920\.7730\.835Low platelet0\.8770\.8800\.846–0\.8750\.8130\.8000\.852High immature platelet fraction0\.8920\.8850\.855–0\.882–––Low immature platelet fraction0\.8700\.8730\.844–0\.851–––High mean platelet volume0\.9020\.9020\.880–0\.907–––Low mean platelet volume0\.7920\.7970\.762–0\.783–––High fibrinogen0\.8910\.8880\.848–0\.8840\.7530\.7130\.794Low fibrinogen0\.9200\.9090\.886–0\.8990\.8660\.8430\.895High partial thromboplastin time0\.8990\.9010\.865–0\.8950\.8670\.8440\.893High international normalized ratio0\.8690\.8740\.834–0\.8630\.8840\.8690\.906High sodium0\.8510\.8490\.813–0\.8400\.8180\.8120\.836Low sodium0\.8540\.8540\.832–0\.8410\.8250\.8110\.834High potassium0\.8360\.8340\.807–0\.8330\.8340\.8150\.845Low potassium0\.8680\.8710\.838–0\.8650\.8190\.8230\.835High glucose0\.8320\.8340\.796–0\.8300\.9270\.9200\.935Low glucose0\.8820\.8840\.855–0\.8730\.7830\.7610\.812High creatinine0\.9140\.9100\.883–0\.8960\.8620\.8220\.868High urea0\.9260\.9280\.896–0\.9090\.8770\.8640\.887Low albumin0\.8780\.8770\.849–0\.8690\.8490\.8240\.872High alanine transaminase0\.8850\.8860\.853–0\.8650\.8000\.8100\.826High aspartate aminotransferase0\.8940\.8960\.862–0\.8780\.8000\.7840\.829High lactate dehydrogenase0\.9130\.9100\.875–0\.8970\.8270\.8100\.854High bilirubin0\.8930\.8850\.862–0\.8790\.8340\.8110\.862High cholesterol0\.9060\.9080\.864–0\.8820\.7850\.7540\.861High triglyceride0\.8760\.8580\.817–0\.8430\.7730\.7700\.830High ferritin0\.8700\.8700\.834–0\.8560\.8330\.8010\.852High creatinine kinase0\.9050\.9110\.862–0\.8930\.8090\.8020\.853High C\-reactive protein0\.8490\.8510\.814–0\.8400\.7930\.7630\.829High erythrocyte sedimentation rate0\.9320\.9360\.884–0\.903–––Low PaO20\.9540\.9580\.943–0\.958–––Low SpO20\.8150\.8170\.766–0\.823–––MedicationsAny antibacterial0\.8640\.8630\.831–0\.854–––Any antifungal0\.9420\.9490\.925–0\.941–––Any chemotherapy0\.9740\.9770\.952–0\.969–––Any antiepileptics0\.8700\.8660\.832–0\.866–––Any glucocorticoid0\.8420\.8420\.806–0\.844–––Dexamethasone0\.8420\.8470\.802–0\.848–––Any opioid0\.8530\.8550\.814–0\.856–––Morphine0\.8360\.8320\.799–0\.837–––Fentanyl0\.8430\.8470\.807–0\.846–––Any inotrope0\.9260\.9300\.880–0\.930–––Clinical outcomesLong length of stay \(≥\\geq7 days\)0\.8340\.8350\.795–0\.8350\.8110\.7890\.835Readmission within 30 days0\.7930\.7940\.775–0\.7860\.8330\.8190\.844Mortality0\.9410\.9420\.905–0\.9370\.8950\.8860\.932aFixed\-Vocab FM was not evaluated in cross\-vocabulary transfer because SEDAR composite event tokens had no learned input embeddings\.

bThe target\-vocabulary reference is a Fixed\-Vocab FM trained directly on the SEDAR vocabulary\.

cThe target\-site reference is a Fixed\-Vocab FM trained directly on MIMIC\. MIMIC cross\-site columns include 36 labels\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise\-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Figure S5\. Sample efficiency curves across in\-domain, cross\-vocabulary, and cross\-site settings for PORTER and Fixed\-Vocab FM

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x7.png)

Curves summarize linear\-probe performance as labeled training examples per task increase\. PORTER is compared with Fixed\-Vocab FM when it can be evaluated\. The cross\-vocabulary panel shows PORTER alone because Fixed\-Vocab FM was not evaluated in this setting\. SEDAR composite event tokens had no learned input embeddings\.

Abbreviations: FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise\-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Table S8\. Full\-shot per\-task AUROC for PORTER and text serialization comparator

TaskPORTERText serialization comparatorTransfusionsPlatelet transfusion0\.9670\.963Red cell transfusion0\.9290\.915ProcedureInvasive intubation0\.9240\.896Gastrostomy tube0\.8850\.912Echocardiogram0\.8940\.882Pulmonary function test0\.9660\.948Lumbar puncture0\.9470\.930Surgery0\.8910\.872Interventional radiology0\.8640\.841ImagingPlain radiography chest0\.8200\.797Ultrasound abdomen0\.8410\.821Computerized tomography chest0\.8680\.845Computerized tomography abdomen0\.8940\.867Computerized tomography head0\.9100\.893MRI head0\.9080\.891MRI whole body0\.9480\.911PET0\.8970\.892Laboratory abnormalityHigh white blood count0\.8340\.804Low white blood count0\.9200\.903High absolute neutrophil count0\.8390\.810Low absolute neutrophil count0\.9340\.917High bands0\.8720\.854High lymphocyte0\.8520\.813Low lymphocyte0\.8940\.874High hemoglobin0\.8610\.848Low hemoglobin0\.8870\.866High mean corpuscular volume0\.8920\.895Low mean corpuscular volume0\.8560\.836High reticulocyte count0\.8980\.878Low reticulocyte count0\.8930\.865High platelet0\.8410\.833Low platelet0\.8770\.877High immature platelet fraction0\.8920\.886Low immature platelet fraction0\.8700\.858High mean platelet volume0\.9020\.906Low mean platelet volume0\.7920\.781High fibrinogen0\.8910\.858Low fibrinogen0\.9200\.900High partial thromboplastin time0\.8990\.882High international normalized ratio0\.8690\.838High sodium0\.8510\.814Low sodium0\.8540\.834High potassium0\.8360\.819Low potassium0\.8680\.849High glucose0\.8320\.804Low glucose0\.8820\.849High creatinine0\.9140\.905High urea0\.9260\.916Low albumin0\.8780\.852High alanine transaminase0\.8850\.861High aspartate aminotransferase0\.8940\.873High lactate dehydrogenase0\.9130\.882High bilirubin0\.8930\.877High cholesterol0\.9060\.874High triglyceride0\.8760\.817High ferritin0\.8700\.846High creatinine kinase0\.9050\.821High C\-reactive protein0\.8490\.823High erythrocyte sedimentation rate0\.9320\.916Low PaO20\.9540\.949Low SpO20\.8150\.779MedicationsAny antibacterial0\.8640\.841Any antifungal0\.9420\.935Any chemotherapy0\.9740\.968Any antiepileptics0\.8700\.844Any glucocorticoid0\.8420\.806Dexamethasone0\.8420\.809Any opioid0\.8530\.821Morphine0\.8360\.799Fentanyl0\.8430\.809Any inotrope0\.9260\.909Clinical outcomesLong length of stay \(≥\\geq7 days\)0\.8340\.812Readmission within 30 days0\.7930\.796Mortality0\.9410\.925Abbreviations: AUROC, area under the receiver operating characteristic curve; LLM, large language model\.\.

### Supplementary Table S9\. Per\-task AUROC for PORTER and its text encoder ablations across in\-domain, cross\-vocabulary, and cross\-site settings

TaskIn\-domain \(SK\)Cross\-vocabulary \(SK\)Cross\-site \(MIMIC\)\*BioLORDQwen3BGE\-M3RandomBioLORDQwen3BGE\-M3RandomBioLORDQwen3BGE\-M3RandomTransfusionsPlatelet transfusion0\.9670\.9680\.9690\.9640\.9440\.9390\.9300\.902––––Red cell transfusion0\.9290\.9210\.9240\.9200\.8910\.8930\.8690\.842––––ProcedureInvasive intubation0\.9240\.9250\.9080\.9200\.8940\.8840\.8480\.813––––Gastrostomy tube0\.8850\.8840\.8840\.8670\.8550\.8420\.7840\.765––––Echocardiogram0\.8940\.8890\.8820\.8890\.8690\.8640\.8440\.800––––Pulmonary function test0\.9660\.9620\.9670\.9580\.9410\.9350\.8970\.838––––Lumbar puncture0\.9470\.9460\.9360\.9380\.9160\.8980\.8660\.816––––Surgery0\.8910\.8810\.8790\.8810\.8500\.8520\.8160\.766––––Interventional radiology0\.8640\.8590\.8550\.8510\.8150\.8180\.7780\.734––––ImagingPlain radiography chest0\.8200\.8200\.8180\.8220\.7870\.7850\.7650\.713––––Ultrasound abdomen0\.8410\.8360\.8360\.8390\.8100\.8020\.7740\.736––––Computerized tomography chest0\.8680\.8600\.8550\.8490\.8180\.8080\.8010\.753––––Computerized tomography abdomen0\.8940\.8910\.8910\.8830\.8240\.8470\.7930\.738––––Computerized tomography head0\.9100\.9120\.9070\.9030\.8690\.8630\.7930\.722––––MRI head0\.9080\.9050\.9060\.9050\.8750\.8550\.8120\.739––––MRI whole body0\.9480\.9400\.9380\.9520\.9030\.8450\.8740\.778––––PET0\.8970\.9170\.9160\.8800\.8230\.8290\.7950\.750––––Laboratory abnormalityHigh white blood count0\.8340\.8330\.8310\.8350\.7810\.7750\.7490\.6970\.7600\.7650\.7560\.758Low white blood count0\.9200\.9190\.9180\.9190\.8830\.8750\.8670\.8470\.8690\.8910\.8760\.875High absolute neutrophil count0\.8390\.8380\.8360\.8380\.7880\.7850\.7540\.7210\.7950\.7850\.7760\.758Low absolute neutrophil count0\.9340\.9340\.9330\.9310\.9060\.9020\.8840\.8640\.8530\.8600\.8500\.856High bands0\.8720\.8700\.8710\.8690\.8450\.8380\.8220\.809––––High lymphocyte0\.8520\.8480\.8490\.8490\.8160\.8170\.7980\.7610\.7130\.7160\.6890\.704Low lymphocyte0\.8940\.8950\.8940\.8920\.8710\.8640\.8490\.8290\.8160\.8030\.8060\.780High hemoglobin0\.8610\.8470\.8440\.8400\.8180\.8090\.7990\.777––––Low hemoglobin0\.8870\.8860\.8820\.8810\.8550\.8560\.8320\.8050\.8590\.8620\.8650\.855High mean corpuscular volume0\.8920\.9040\.8980\.8870\.8600\.8540\.8520\.8250\.7930\.8080\.8080\.812Low mean corpuscular volume0\.8560\.8660\.8630\.8480\.7790\.7810\.7650\.7240\.8060\.8230\.8230\.821High reticulocyte count0\.8980\.8900\.8910\.8940\.8550\.8550\.8460\.824––––Low reticulocyte count0\.8930\.8760\.8860\.8720\.8510\.8390\.8210\.793––––High platelet0\.8410\.8340\.8360\.8410\.8050\.7990\.7840\.7530\.7920\.7920\.7880\.773Low platelet0\.8770\.8760\.8790\.8880\.8460\.8380\.8250\.7990\.8130\.8210\.8080\.808High immature platelet fraction0\.8920\.8840\.8890\.8910\.8550\.8530\.8440\.820––––Low immature platelet fraction0\.8700\.8610\.8680\.8590\.8440\.8270\.8250\.794––––High mean platelet volume0\.9020\.9000\.9030\.9010\.8800\.8760\.8650\.835––––Low mean platelet volume0\.7920\.7780\.7960\.8020\.7620\.7540\.7350\.694––––High fibrinogen0\.8910\.8940\.8920\.8820\.8480\.8320\.8230\.7740\.7530\.7700\.7600\.742Low fibrinogen0\.9200\.9130\.9040\.9070\.8860\.8780\.8400\.8230\.8660\.8690\.8750\.867High partial thromboplastin time0\.8990\.8990\.9010\.9020\.8650\.8690\.8360\.8080\.8670\.8630\.8610\.847High international normalized ratio0\.8690\.8710\.8680\.8700\.8340\.8260\.7990\.7690\.8840\.8820\.8830\.873High sodium0\.8510\.8450\.8410\.8420\.8130\.8060\.7860\.7500\.8180\.8190\.8190\.810Low sodium0\.8540\.8550\.8510\.8490\.8320\.8090\.7970\.7780\.8250\.8150\.8210\.799High potassium0\.8360\.8290\.8330\.8290\.8070\.7970\.7850\.7640\.8340\.8280\.8280\.816Low potassium0\.8680\.8680\.8630\.8640\.8380\.8380\.8180\.7780\.8190\.8110\.8080\.805High glucose0\.8320\.8340\.8330\.8320\.7960\.7960\.7740\.7500\.9270\.9250\.9240\.917Low glucose0\.8820\.8810\.8810\.8760\.8550\.8360\.8300\.8010\.7830\.7880\.7860\.773High creatinine0\.9140\.9160\.9120\.9130\.8830\.8630\.8340\.7990\.8620\.8500\.8470\.817High urea0\.9260\.9240\.9300\.9240\.8960\.8890\.8670\.8290\.8770\.8720\.8730\.851Low albumin0\.8780\.8750\.8740\.8730\.8490\.8460\.8230\.8040\.8490\.8490\.8430\.842High alanine transaminase0\.8850\.8850\.8860\.8820\.8530\.8310\.8080\.7890\.8000\.8100\.8030\.794High aspartate aminotransferase0\.8940\.8990\.8940\.8940\.8620\.8360\.8160\.7930\.8000\.8070\.7970\.784High lactate dehydrogenase0\.9130\.9100\.9060\.9050\.8750\.8550\.8430\.8410\.8270\.8290\.8300\.821High bilirubin0\.8930\.8900\.8890\.8920\.8620\.8550\.8340\.7980\.8340\.8530\.8360\.824High cholesterol0\.9060\.8900\.8990\.9070\.8640\.8230\.8330\.7020\.7850\.7820\.7900\.789High triglyceride0\.8760\.8610\.8450\.8620\.8170\.7980\.7900\.7280\.7730\.7740\.7680\.777High ferritin0\.8700\.8650\.8640\.8710\.8340\.8330\.8110\.7850\.8330\.8300\.8310\.819High creatinine kinase0\.9050\.9010\.9040\.9120\.8620\.8530\.8170\.7550\.8090\.8140\.8060\.804High C\-reactive protein0\.8490\.8530\.8510\.8550\.8140\.8070\.7850\.7420\.7930\.7860\.7990\.758High erythrocyte sedimentation rate0\.9320\.9340\.9330\.9360\.8840\.8850\.8560\.823––––Low PaO20\.9540\.9560\.9540\.9520\.9430\.9460\.9200\.901––––Low SpO20\.8150\.8100\.8040\.8050\.7660\.7640\.7390\.684––––MedicationsAny antibacterial0\.8640\.8630\.8620\.8580\.8310\.8170\.7820\.726––––Any antifungal0\.9420\.9450\.9450\.9340\.9250\.9190\.9040\.884––––Any chemotherapy0\.9740\.9680\.9720\.9660\.9520\.9480\.9380\.915––––Any antiepileptics0\.8700\.8620\.8620\.8600\.8320\.8210\.7910\.733––––Any glucocorticoid0\.8420\.8410\.8390\.8360\.8060\.7880\.7530\.719––––Dexamethasone0\.8420\.8390\.8440\.8370\.8020\.7970\.7560\.725––––Any opioid0\.8530\.8450\.8450\.8400\.8140\.8070\.7840\.727––––Morphine0\.8360\.8300\.8260\.8230\.7990\.7860\.7570\.708––––Fentanyl0\.8430\.8370\.8370\.8390\.8070\.7960\.7780\.733––––Any inotrope0\.9260\.9330\.9370\.9220\.8800\.8960\.8390\.810––––Clinical outcomesLong length of stay \(≥\\geq7 days\)0\.8340\.8280\.8270\.8300\.7950\.7780\.7620\.7240\.8110\.8140\.8070\.796Readmission within 30 days0\.7930\.7930\.7940\.7890\.7750\.7770\.7680\.7440\.8330\.8260\.8320\.814Mortality0\.9410\.9350\.9400\.9310\.9050\.8780\.9030\.8280\.8950\.8920\.9000\.884\*MIMIC cross\-site columns include 36 labels\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; MIMIC, Medical Information Mart for Intensive Care; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Figure S6\. Sample efficiency curves across in\-domain, cross\-vocabulary, and cross\-site settings for PORTER and its text encoder ablations

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x8.png)

Curves summarize linear probe performance as labeled training examples per task increase\. PORTER is compared against three ablations that hold the FiLM numeric pathway fixed while varying the text encoder: PORTER\-Qwen3 \(Qwen3\-Embedding\-8B\), PORTER\-BGE\-M3 \(BGE\-M3\), and PORTER\-Random \(randomly initialized encoder\)\.

Abbreviations: FiLM, feature\-wise linear modulation; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise\-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Figure S7\. Patient representation geometry under cross\-vocabulary transfer for PORTER and its text encoder ablations

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x9.png)

For each encoder, patient representations were computed for the same 9,585 unique test patients using OMOP\-derived event descriptions and SEDAR\-derived event descriptions\. Each point plots the cosine distance between a pair of patients under the OMOP\-derived representation against the corresponding distance under the SEDAR\-derived representation\. The scatter plots show a random subsample \(200,000 pairs\) for visualization\. Spearmanρ\\rhoand the OLS fit summarize how well the encoder preserves patient\-level geometry\. Higherρ\\rho, slope closer to 1, and intercept closer to 0 indicate stronger preservation of patient\-level geometry under vocabulary transfer\.

Abbreviations: OLS, ordinary least squares; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise\-wide Data in Azure Repository\.

### Supplementary Table S10\. Per\-task AUROC for PORTER and its numeric pathway ablations across in\-domain, cross\-vocabulary, and cross\-site settings

TaskIn\-domain \(SK\)Cross\-vocabulary \(SK\)Cross\-site \(MIMIC\)\*PORTERPORTER\-NoNumPORTER\-NumTextPORTERPORTER\-NoNumPORTER\-NumTextPORTERPORTER\-NoNumPORTER\-NumTextTransfusionsPlatelet transfusion0\.9670\.9630\.9680\.9440\.9460\.942–––Red cell transfusion0\.9290\.9120\.9190\.8910\.8800\.881–––ProcedureInvasive intubation0\.9240\.9350\.9190\.8940\.8850\.885–––Gastrostomy tube0\.8850\.8720\.8620\.8550\.7950\.854–––Echocardiogram0\.8940\.8870\.8930\.8690\.8640\.856–––Pulmonary function test0\.9660\.9650\.9650\.9410\.9290\.928–––Lumbar puncture0\.9470\.9340\.9500\.9160\.9140\.909–––Surgery0\.8910\.8880\.8850\.8500\.8590\.844–––Interventional radiology0\.8640\.8580\.8580\.8150\.8120\.815–––ImagingPlain radiography chest0\.8200\.8050\.8180\.7870\.7590\.767–––Ultrasound abdomen0\.8410\.8360\.8430\.8100\.7990\.802–––Computerized tomography chest0\.8680\.8600\.8640\.8180\.8150\.818–––Computerized tomography abdomen0\.8940\.8780\.8850\.8240\.8390\.832–––Computerized tomography head0\.9100\.9090\.9140\.8690\.8850\.870–––MRI head0\.9080\.9080\.9060\.8750\.8770\.867–––MRI whole body0\.9480\.9420\.9330\.9030\.8460\.860–––PET0\.8970\.8940\.9090\.8230\.9120\.803–––Laboratory abnormalityHigh white blood count0\.8340\.7940\.8240\.7810\.7400\.7730\.7600\.7470\.745Low white blood count0\.9200\.8880\.9160\.8830\.8640\.8780\.8690\.8440\.844High absolute neutrophil count0\.8390\.8090\.8300\.7880\.7610\.7880\.7950\.7880\.782Low absolute neutrophil count0\.9340\.9150\.9350\.9060\.8980\.9080\.8530\.8440\.822High bands0\.8720\.8640\.8680\.8450\.8300\.834–––High lymphocyte0\.8520\.8220\.8460\.8160\.7930\.8030\.7130\.6940\.682Low lymphocyte0\.8940\.8840\.8920\.8710\.8590\.8640\.8160\.8050\.791High hemoglobin0\.8610\.8370\.8500\.8180\.8020\.791–––Low hemoglobin0\.8870\.8670\.8800\.8550\.8300\.8410\.8590\.8470\.851High mean corpuscular volume0\.8920\.8690\.8910\.8600\.8350\.8450\.7930\.7550\.752Low mean corpuscular volume0\.8560\.8040\.8560\.7790\.7590\.7650\.8060\.6800\.663High reticulocyte count0\.8980\.8890\.8890\.8550\.8430\.848–––Low reticulocyte count0\.8930\.8640\.8770\.8510\.8260\.834–––High platelet0\.8410\.8180\.8410\.8050\.7790\.7920\.7920\.7700\.775Low platelet0\.8770\.8680\.8860\.8460\.8330\.8400\.8130\.7980\.800High immature platelet fraction0\.8920\.8840\.8930\.8550\.8520\.854–––Low immature platelet fraction0\.8700\.8620\.8720\.8440\.8410\.837–––High mean platelet volume0\.9020\.8940\.9020\.8800\.8700\.867–––Low mean platelet volume0\.7920\.7680\.7960\.7620\.7330\.755–––High fibrinogen0\.8910\.8740\.8850\.8480\.8450\.8360\.7530\.7440\.719Low fibrinogen0\.9200\.9120\.9080\.8860\.8790\.8600\.8660\.8440\.852High partial thromboplastin time0\.8990\.8920\.8990\.8650\.8530\.8620\.8670\.8520\.854High international normalized ratio0\.8690\.8600\.8630\.8340\.8200\.8260\.8840\.8760\.877High sodium0\.8510\.8360\.8460\.8130\.7990\.8050\.8180\.7950\.798Low sodium0\.8540\.8380\.8510\.8320\.8050\.8240\.8250\.7980\.800High potassium0\.8360\.8260\.8310\.8070\.7920\.7990\.8340\.8170\.821Low potassium0\.8680\.8620\.8690\.8380\.8310\.8320\.8190\.8070\.800High glucose0\.8320\.8240\.8310\.7960\.7880\.7890\.9270\.9240\.925Low glucose0\.8820\.8780\.8770\.8550\.8390\.8510\.7830\.7770\.760High creatinine0\.9140\.8970\.9010\.8830\.8610\.8750\.8620\.8390\.833High urea0\.9260\.9160\.9260\.8960\.8920\.8890\.8770\.8600\.860Low albumin0\.8780\.8700\.8740\.8490\.8430\.8410\.8490\.8390\.836High alanine transaminase0\.8850\.8700\.8830\.8530\.8350\.8390\.8000\.7830\.780High aspartate aminotransferase0\.8940\.8820\.8950\.8620\.8430\.8540\.8000\.7810\.773High lactate dehydrogenase0\.9130\.9020\.8980\.8750\.8680\.8600\.8270\.8110\.806High bilirubin0\.8930\.8870\.8840\.8620\.8500\.8550\.8340\.8000\.814High cholesterol0\.9060\.8830\.8980\.8640\.8090\.8550\.7850\.7560\.804High triglyceride0\.8760\.8650\.8580\.8170\.8010\.7720\.7730\.7580\.753High ferritin0\.8700\.8520\.8560\.8340\.8310\.8250\.8330\.8080\.802High creatinine kinase0\.9050\.8990\.9030\.8620\.8380\.8450\.8090\.7830\.797High C\-reactive protein0\.8490\.8380\.8440\.8140\.7930\.8020\.7930\.7900\.781High erythrocyte sedimentation rate0\.9320\.9140\.9290\.8840\.8800\.890–––Low PaO20\.9540\.9600\.9560\.9430\.9460\.948–––Low SpO20\.8150\.8040\.8130\.7660\.7630\.765–––MedicationsAny antibacterial0\.8640\.8580\.8610\.8310\.8230\.816–––Any antifungal0\.9420\.9450\.9430\.9250\.9230\.917–––Any chemotherapy0\.9740\.9740\.9730\.9520\.9510\.953–––Any antiepileptics0\.8700\.8630\.8640\.8320\.8160\.813–––Any glucocorticoid0\.8420\.8360\.8400\.8060\.8000\.797–––Dexamethasone0\.8420\.8360\.8470\.8020\.8020\.794–––Any opioid0\.8530\.8450\.8490\.8140\.8140\.808–––Morphine0\.8360\.8340\.8360\.7990\.7910\.785–––Fentanyl0\.8430\.8370\.8410\.8070\.8080\.803–––Any inotrope0\.9260\.9310\.9240\.8800\.8960\.890–––Clinical outcomesLong length of stay \(≥\\geq7 days\)0\.8340\.8230\.8310\.7950\.7770\.7820\.8110\.8050\.798Readmission within 30 days0\.7930\.7910\.7900\.7750\.7780\.7750\.8330\.8360\.825Mortality0\.9410\.9290\.9320\.9050\.8890\.8730\.8950\.8770\.868\*MIMIC cross\-site columns include 36 labels\.

Abbreviations: AUROC, area under the receiver operating characteristic curve; MIMIC, Medical Information Mart for Intensive Care; NoNum, no numeric pathway; NumText, numeric\-as\-text; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Figure S8\. Sample efficiency curves across in\-domain, cross\-vocabulary, and cross\-site settings for PORTER and its numeric pathway ablations

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x10.png)

Curves summarize linear probe performance as labeled training examples per task increase\. PORTER is compared against two numeric pathway ablations while holding the text encoder fixed: PORTER\-NumText \(numeric value and reference range added to the event text description instead of using FiLM\) and PORTER\-NoNum \(no numeric representation\)\.

Abbreviations: FiLM, feature\-wise linear modulation; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise\-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids\.

### Supplementary Figure S9\. Synonym invariance and numeric sensitivity of event representations passed to the transformer for PORTER and its numeric pathway ablations

![[Uncaptioned image]](https://arxiv.org/html/2606.24102v1/x11.png)

Top row: cosine similarity between SEDAR\-derived descriptions and OMOP\-derived event descriptions, shown as half\-violins per domain \(within\-concept on the left, across\-concept distractors on the right\)\. Bottom row: Spearmanρ\\rhofor each measurement concept between decile index and cosine distance from the first decile, shown as box plots with jittered per\-concept points\. Lines connect the same measurement concept across the PORTER\-NumText and PORTER columns\. Line color indicates whether FiLM improves or worsens numeric sensitivity relative to NumText, as shown in the legend\. PORTER\-NoNum does not consider numeric values, so embeddings across deciles are identical for each concept andρ\\rhois undefined\.

Abbreviations: FiLM, feature\-wise linear modulation; NoNum, no numeric pathway; NumText, numeric values rendered as text; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise\-wide Data in Azure Repository\.