arXiv:2606.24102v1 [cs.CL] 23 Jun 2026
PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models
Lin Lawrence Guo Child Health Evaluative Sciences The Hospital for Sick Children Toronto, Canada
Adam Paul Yan Child Health Evaluative Sciences Division of Haematology/Oncology The Hospital for Sick Children Toronto, Canada
Emily Vettese Child Health Evaluative Sciences The Hospital for Sick Children Toronto, Canada
Lillian Sung∗ Child Health Evaluative Sciences Division of Haematology/Oncology The Hospital for Sick Children Toronto, Canada
Abstract Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values. This limits transfer across institutions and even across deployment pipelines within the same institution. We introduce PORTER (Portable EHR Representations), a languagegrounded structured EHR foundation model that decouples event representation from this fixed vocabulary. PORTER represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines with an autoregressively pretrained temporal backbone. Across 74 clinical prediction tasks at a pediatric hospital, PORTER matched the mean area-underthe-receiver-operating-characteristic curve (AUROC) of a fixed-vocabulary model with the same temporal backbone and pretraining objective. When the same patient timelines were rendered using event descriptions not seen during pretraining, PORTER transferred without retraining or vocabulary mapping, recovering 97.1% of the mean AUROC of a model trained directly on the target vocabulary. When transferred to MIMIC, PORTER outperformed the fixed-vocabulary model, which dropped 69% of events because their tokens were unseen. Mechanistic analyses showed cross-vocabulary transfer tracked preservation of patient-level representation geometry rather than the scale of the text encoder, and the numeric pathway improved sensitivity to magnitude without disrupting clinical concept identity. PORTER also achieved higher AUROC than a task-specific text serialization comparator, at 329-fold lower amortized compute. PORTER is a step toward vocabulary-independent EHR foundation models that reduce the need for vocabulary harmonization while preserving in-domain performance and enabling efficient cross-task reuse.
1
INTRODUCTION
Structured electronic health record (EHR) foundation models learn reusable patient representations through self-supervised pretraining on longitudinal sequences of clinical events.[1–8] Each event is a timestamped ∗
Corresponding author: [email protected]
Portable EHR Representations
Preprint
occurrence of a clinical concept, such as a diagnosis or laboratory result, and may include structured attributes such as numeric values. These models have demonstrated robustness under temporal dataset shift[9] and transfer across populations[10] and institutions[11], although such transfer generally assumes that evaluation data can be encoded using event-token representations learned during pretraining. In practice, the codes and descriptions used to represent clinical events vary across institutions, coding standards, and data pipelines. Most structured EHR foundation models convert each clinical event into a discrete event token from a fixed vocabulary learned during pretraining.[12] These tokens may represent a clinical concept alone or a composite of the clinical concept and structured attributes, such as a discretized numeric bin.[13] When deployment data include new clinical concepts, different local codes or names for the same concepts, or new concept-attribute combinations, those events produce tokens for which the model has no learned representation. Even within a single institution, models may be developed using retrospective datasets mapped to common data models such as the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM), then deployed using production feeds with different local codes or naming conventions. Therefore, vocabulary shift can arise even when the patient population and clinical setting remain the same. A portable structured EHR foundation model must preserve the meaning of a clinical event even when that event is described using different codes or descriptions. Standard vocabularies and common data models reduce this heterogeneity, but mapping to them is labor-intensive, error-prone, and does not eliminate differences in concept coverage or local implementation. Numeric values introduce a second representation challenge. For measurement events, the clinical concept identifies what was measured, while the numeric value and reference range determine magnitude and abnormality. A portable model must therefore represent both concept semantics and numeric values without tying either to a fixed token vocabulary. Text-based representations have emerged as a way to reduce the dependence of structured EHR models on fixed vocabularies and manual harmonization. One line of work serializes patient timelines into natural language and uses a large language model or text encoder to produce patient-level representations.[14–16] This approach avoids a fixed event-token vocabulary, but requires the model to infer concept semantics, temporal relationships, and numeric meaning from the serialized text.[17] When serialization is task-specific, patient representations must also be recomputed for each downstream task. A second line of work operates at the code or event level by representing clinical concepts through their descriptions or by learning aligned code representations across sites.[18–23] These methods have shown that text-based code or event representations can improve portability across vocabularies, schemas, institutions, and languages. They differ in whether the text encoder is frozen or updated during training, which determines how much an event description's representation is shaped by the training vocabulary. A frozen pretrained encoder applies the same semantic mapping to descriptions from any vocabulary, including vocabularies not seen during training, without modifying model weights[18]. Some also use self-supervised objectives, but as initialization for supervised finetuning rather than to learn task-agnostic patient representations reused across downstream tasks.[22] Across existing frameworks, numeric values are commonly discarded, rendered as text, discretized, or represented through token-level schemes. Existing approaches therefore have not fully combined vocabulary-independent inputs, autoregressive pretraining for reusable task-agnostic patient representations, and explicit representation of numeric values within a single structured EHR foundation model (Supplementary Table S1). We introduce PORTER (Portable EHR Representations), a language-grounded structured EHR foundation model that separates concept semantics, numeric values, and temporal dynamics into distinct components. Rather than serializing full patient histories into text, PORTER applies language grounding at the clinical event level. Each event is paired with a natural-language description of its underlying clinical concept, which is processed by a frozen text encoder to provide vocabulary-independent concept representations. Because this text encoder is applied once per unique event description rather than repeatedly across patient histories, these representations can be cached and reused across patients and downstream tasks. For events containing numeric values, a separate learned pathway encodes numeric magnitude and relative abnormality directly, rather than rendering numeric values as text. Feature-wise linear modulation (FiLM)[24] then uses this numeric information to modulate the text-derived concept representation and produce the event input representation. Finally, a temporal backbone learns clinical dynamics from sequences of these event input representations through autoregressive pretraining. After pretraining, the backbone is frozen and produces reusable patient representations that downstream tasks use through linear probes. At inference, PORTER can represent clinical concepts from vocabularies not seen during pretraining without retraining or explicit vocabulary mapping, provided they can be rendered as interpretable event descriptions.
2
Portable EHR Representations
Preprint
We evaluate PORTER across three settings using a fixed-vocabulary EHR foundation model (Fixed-Vocab FM) matched on backbone architecture, pretraining objective, and training schedule as the primary comparator. First, we test whether PORTER matches Fixed-Vocab FM in-domain across 74 clinical prediction tasks. Second, we evaluate cross-vocabulary transfer within the same institution by applying pretrained PORTER to patient timelines where event descriptions are derived from institutional EHR terminology rather than the OMOP-derived descriptions used during pretraining. This setting isolates vocabulary shift while holding patient timelines fixed and cannot be directly supported by Fixed-Vocab FM without vocabulary mapping or retraining. Third, we evaluate cross-site transfer to MIMIC, where vocabulary, population, and clinical setting differ from pretraining. We additionally ablate the numeric pathway and text encoder choice across evaluation settings and compare PORTER with a patient-level text serialization comparator. This study makes the following contributions: • We introduce PORTER, a language-grounded structured EHR foundation model that uses eventdescription inputs instead of fixed-vocabulary input embeddings. PORTER pairs a frozen pretrained text encoder, which yields vocabulary-independent concept representations that are cached and reused, with a dedicated numeric pathway that integrates magnitude through FiLM, and learns clinical dynamics with an autoregressive temporal backbone that is frozen after pretraining and reused across tasks through linear probes. • We evaluate PORTER using a controlled cross-vocabulary design that holds patients and downstream task labels fixed while changing only the event-description naming system. PORTER transferred to unseen event descriptions without retraining or vocabulary mapping, recovering 97.1% of the AUROC of a target-vocabulary reference model. Text encoder ablations showed that cross-vocabulary performance tracked preservation of patient-level representation geometry rather than encoder scale. • PORTER matched an architecture-matched Fixed-Vocab FM across 74 in-domain tasks and improved transfer on 31 of 36 tasks at an external site, where Fixed-Vocab FM dropped 69% of events because their tokens were unseen. • PORTER’s dedicated numeric pathway improved sensitivity to numeric magnitude compared with rendering values as text, while preserving clinical concept identity. • Compared with a task-specific patient-level text serialization comparator, PORTER achieved higher AUROC on 69 of 74 tasks, with lower amortized compute as reuse of task-agnostic patient representations increased.
2
METHODS
2.1
Hospital Datasets
This study used EHR data from The Hospital for Sick Children (SickKids), a tertiary pediatric hospital, as the primary development site, and MIMIC, derived from Beth Israel Deaconess Medical Center (BIDMC), an adult academic medical center, for external evaluation. SickKids uses Epic (Epic Systems, Verona, WI) as its enterprise EHR. The SickKids dataset was sourced from the SickKids Enterprise-wide Data in Azure Repository (SEDAR)[25], which consolidates EHR data from SickKids' Epic Clarity database into a clinically oriented, validated and standardized schema. EHR data were mapped to the Medical Event Data Standard (MEDS)[26, 27] format with clinical concepts standardized to Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) ontologies. The MIMIC dataset (MIMIC-IV, version 1.0)[28] contains de-identified EHR data from patients admitted to the intensive care unit or emergency department at BIDMC between 2008 and 2019. MIMIC data were mapped to the OMOP CDM using code provided by the Observational Health Data Sciences and Informatics MIMIC project[29] and subsequently converted to MEDS format. As part of MIMIC's de-identification process, patient timelines are shifted to an anchor year within a three-year window. To support consistent temporal splitting across SickKids and MIMIC, we deterministically assigned each patient a representative calendar year within their anchor group via hashing of the patient identifier. Use of SEDAR data for this study was approved by the Research Ethics Board (REB) at SickKids (REB number: 1000074527). Use of MIMIC was approved under the oversight of the Institutional Review Board at BIDMC and is made publicly available on PhysioNet.[30]
3
Portable EHR Representations
2.2
Preprint
Cohort Definition and Splitting
The cohort selection process is summarized in Supplementary Figure S1. Pretraining cohorts were defined at the patient level. For SickKids, we included all patients in SEDAR over the study period, with clinical events spanning June 2, 2018 (EHR go-live) through April 7, 2026. Patients were excluded if they had missing date of birth. For MIMIC, all patients in the dataset were included. Within each dataset, patients were deterministically assigned to training (~90%) and validation (~10%) subsets via hashing of the patient identifier. For SickKids, pretraining used events through December 31, 2024 for training patients and through March 31, 2025 for validation patients. For MIMIC, pretraining used events through December 31, 2016 for training patients and through December 31, 2017 for validation patients. Downstream evaluation cohorts were defined at the admission level. For SickKids, we included inpatient admissions where age at the prediction time was 28 days or older. For MIMIC, we included inpatient admissions where age at prediction time was 18 years or older. Admissions were assigned to training, validation, and test sets according to the prediction time, using the same calendar cutoffs applied to the corresponding pretraining cohorts. Patients with multiple admissions could contribute admissions to different temporal periods, reflecting a deployment setting in which previously observed patients may return for future admissions. 2.3
Clinical Prediction Tasks
The evaluation tasks were adapted from our previous study.[13] At SickKids, we evaluated 74 clinical prediction tasks spanning six task families: transfusions, procedures, imaging, laboratory abnormalities, medication administrations, and clinical outcomes, with laboratory abnormalities defined using site-specific reference ranges. For MIMIC, we evaluated 36 tasks comprising an adapted subset of the SickKids clinical outcome and laboratory abnormality tasks, with laboratory abnormalities defined using MIMIC-specific reference ranges. Prediction time was set to the end of the admission day at 23:59 for all tasks except 30-day readmission, for which it was set at 23:59 on the day before discharge. The prediction window extended until discharge for all tasks except long length of stay and readmission, which used fixed windows of 7 days after admission and 30 days after discharge, respectively. For each task, admissions in which death, discharge, or outcome occurred on or before the prediction time were excluded. The full task list and cohort sizes are provided in Supplementary Table S2. 2.4
PORTER Architecture
We define a clinical event as a timestamped EHR occurrence associated with a clinical concept, a naturallanguage event description, and, where applicable, numeric metadata such as value and reference range. Each event contributes one input representation to the temporal model. In contrast to fixed-vocabulary models that represent events by indexing a learned embedding table over discrete event tokens, PORTER generates event representations directly from these event descriptions and numeric metadata, making the input pathway vocabulary-independent. PORTER comprises three components (Figure 1). First, a frozen text encoder converts each event description to a vocabulary-independent concept representation, which is cached and projected to the transformer hidden dimension. Second, for events with numeric measurements, a numeric pathway encodes measurement magnitude and relative abnormality, and feature-wise linear modulation (FiLM) integrates this numeric information by scaling and shifting the projected text-derived representation. Third, a temporal transformer backbone processes the fused event representations and learns longitudinal clinical dynamics through autoregressive next-event prediction. The following sections describe these components and the pretraining objective in detail. 2.4.1
Language-Grounded Event Representation
Each clinical event was represented by a templated event description, with numeric metadata retained for separate numeric encoding. Templates spanned nine categories: demographics, measurement, observation, condition, procedure, drug, note, specimen, and visit. Where relevant, templates incorporated additional structured EHR fields, including non-numeric measurement result categories and drug route. For example, measurement events were represented as "Measurement: {concept name}" or "Measurement: {concept name}. Result: {non-numeric result concept name}." For the primary PORTER setup, both the primary concept and attribute slots were populated using OMOP concept names. For cross-vocabulary evaluation, we constructed 4
Portable EHR Representations
Preprint
B Evaluation Framework
A PORTER Architecture Training Time
Non-Numeric Event (Description Pathway Only) Numeric Event (FiLM Pathway)
Embedding Space
Febrile Seizure WBC …
)
Inference Time
Text Encoder (Pretrained)
Febrile Seizure ≈ Convulsion
WBC ≈ Leukocyte
Febrile Convulsion Leukocyte Count
Birth
Sex: F
ED Visit
Temperature 103 F
Febrile Seizure
WBC 13.2 x 109 / L
Day 0
Day 0
Day 244.2
Day 245.1
Day 245.4
Day 245.8
Unrelated Concepts
Event Text Description Description Encoder Embedding (etext) Offline, cached per vocabulary etext
xnum ∈ ℝ 4
Cache PORTER Patient Representation
x T events in patient timeline
γ
Setting
Causal self-attention +
In-domain
RoPE on days since birth
SickKids; 74 tasks
Feed-forward network
Cross-vocabulary
FiLM
Same patients and tasks
+
β hfused Numeric Pathway
Description Pathway
Task-Specific Linear Probe
Temporal Backbone 115M Parameters
htext
Prediction Time
Patient Timeline
Cross-site
Softmax
OMOP; 36 tasks
P( eventt+1 | h ≤ t )
Prediction
Pretrain
Evaluate
FixedVocab FM
PORTER
OMOP
OMOP
✓
✓
OMOP
SEDAR
✗
✓
SickKids (OMOP)
MIMIC (OMOP)
✓*
✓
Fixed-Vocab FM: learned embeddings for fixed event tokens PORTER: any event with an interpretable description can be represented ✓*: Requires harmonization (OMOP). Unseen tokens dropped.
Next-event prediction head discarded after selfsupervised pretraining.
Figure 1: PORTER architecture and evaluation framework. (A) PORTER represents each clinical event through an event-description pathway and, when applicable, a numeric pathway. A frozen pretrained text encoder, BioLORD, maps event descriptions to dense embeddings, etext , which are projected into the backbone hidden dimension, htext . For events with numeric values, a parallel numeric pathway encodes normalized numeric features through a learned multilayer perceptron and modulates the text-derived concept representation using feature-wise linear modulation, producing a fused event input representation, hfused . Numeric values are normalized using institutional reference ranges where available, with log-transformed raw values as a fallback. Three binary indicators specify whether a numeric value is present and which normalization was applied. Event-description embeddings are computed once per unique event description and cached offline. A causal transformer backbone with 115M parameters and rotary position embeddings applied to patient age in days is pretrained using self-supervised next-event prediction over patient event sequences. The prediction head is discarded after pretraining. (B) Downstream evaluation extracts frozen patient representations at a task-specific prediction time and trains a task-specific linear probe. Three evaluation settings test increasing portability demands. In-domain evaluation uses the same institution and OMOP-derived event descriptions. Cross-vocabulary evaluation uses the same patients and labels but SEDAR-derived event descriptions. Cross-site evaluation transfers from SickKids to MIMIC. A conventional fixed-vocabulary foundation model, Fixed-Vocab FM, serves as the primary comparator. Fixed-Vocab FM can operate in-domain and can be evaluated cross-site when data are harmonized to OMOP, although unseen event tokens are dropped. It cannot directly support cross-vocabulary evaluation without vocabulary mapping or retraining. Abbreviations: PORTER, Portable EHR Representations; MLP, multilayer perceptron; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise-wide Data in Azure Repository; MIMIC, Medical Information Mart for Intensive Care; FiLM, feature-wise linear modulation; EHR, electronic health records; RoPE, rotary position embedding. a separate set of event descriptions from analogous SEDAR concept and attribute names, allowing the same patient histories to be represented using institutional rather than OMOP terminology. Full templates and source columns are provided in Supplementary Table S3. Each event description was encoded once using a frozen text encoder and cached as a static event-description embedding table. We evaluated BioLORD2023[31], BGE-M3[32], and Qwen3-Embedding-8B[33] as candidate encoders and selected BioLORD-2023 for the primary PORTER experiments based on the lowest pretraining validation loss (see Supplementary Figure S2 for pretraining loss curves across text encoder variants). Cached embeddings were projected to the backbone hidden dimension using a two-layer multilayer perceptron with GELU activation.
5
Portable EHR Representations
2.4.2
Preprint
Numeric Pathway via FiLM
For events with numeric values, PORTER constructs a fixed feature vector capturing measurement magnitude and, when available, relative abnormality without per-concept population statistics. When a valid reference range was available, the numeric scalar encoded the value’s position within the range, with 0 corresponding to the lower limit and 1 to the upper limit, clipped to [-2, 3]. These asymmetric bounds reflect the heavier upper tail of clinical lab values while retaining at least 99% of SickKids training data. When no reference range was available, PORTER used a log-magnitude fallback, log(1 + |v|) / 7, clipped to [0, 1.5]. The divisor of 7 maps log(1 + 1000) ≈ 6.9 to approximately 1.0. In SickKids training data, the 99th and 99.9th percentiles of the resulting pre-clip scalar were 0.83 and 1.32, respectively. Supplementary Figure S3 provides the empirical distribution of pre-clip numeric scalar features. The numeric feature vector contained this scalar, indicators for reference-range scaling and log-magnitude fallback, and an indicator for numeric value presence. Exactly one scaling indicator was active when a numeric value was present. For events without numeric content, all indicators and the scalar were set to zero. This design preserves a fixed numeric feature layout across institutions and vocabularies without relying on concept-specific distributional summaries, such as means, standard deviations, or quantiles. The feature vector was passed through a small multilayer perceptron with two 128-dimensional fully connected layers and GELU activations. Two linear heads then produced a residual scaling term γ and a shift term β, each with dimension equal to the transformer hidden size. FiLM was applied to the projected text embedding htext as: hfused = (1 + γ) ⊙ htext + β The γ and β heads were initialized to zero so that the fused representation was initially equal to the text representation. For events without numeric content, the numeric feature vector was all zeros, and the pathway was gated off by forcing γ = 0 and β = 0, yielding hfused = htext . 2.4.3
Backbone Transformer
The backbone was adapted from the architecture and hyperparameters used in our prior study[13]. It was a decoder-only transformer[34, 35] with 28 layers, a hidden dimension of 768, and 12 attention heads, yielding 115.6 million backbone parameters. Temporal information was encoded using rotary position embeddings (RoPE)[36] applied to patient age in days, so relative positional rotations reflected elapsed time between events. Transformer layers alternated between global attention and local attention[37] with a window size of 128 tokens. Each clinical event contributed one position to the input sequence, corresponding to the event representation passed to the backbone. For numeric events, this representation was the text-derived event representation modulated by the numeric pathway, whereas for non-numeric events, it was the projected text-derived representation. Training batches were constructed using event-budget-based sequence packing, with a fixed budget of 16,384 clinical events per batch. Longer patient timelines were truncated, while shorter timelines were packed together using causal masking and patient-boundary masking to prevent cross-patient attention. The maximum effective context window was therefore 16,384 clinical events per patient. The model was implemented in PyTorch 2.7. 2.4.4
Self-Supervised Pretraining
PORTER was pretrained on the SickKids training split using autoregressive next-event prediction, implemented as classification of the next event into a finite output vocabulary of composite discrete tokens observed during pretraining. Following our prior work[13], each composite token combined the primary OMOP concept ID with discretized event attributes, including numeric values discretized into 10 quantile bins where applicable. In PORTER, these composite tokens served only as output labels for autoregressive pretraining, while model inputs were language-grounded event representations. The output head was therefore discarded before downstream representation extraction. Thus, downstream portability depends on the frozen input pathway and temporal backbone, not on reuse of the pretraining output vocabulary. Training used AdamW[38] with a peak learning rate of 5×10-4 , cosine decay with warmup, weight decay 0.05, gradient clipping at 1.0, Adam β 1 = 0.9, Adam β 2 = 0.95, and bfloat16 mixed precision. Models were trained for five epochs on a single NVIDIA H100 or L40S GPU without early stopping.[39] Complete optimizer and training hyperparameters are provided in Supplementary Table S4.
6
Portable EHR Representations
2.5
Preprint
Comparison Models
We compared PORTER against two comparators: a fixed-vocabulary foundation model representing the standard discrete-token input approach[1, 2], and a patient-level text serialization comparator representing task-specific text serialization. The Fixed-Vocab FM served as a controlled comparator that differed from PORTER in how clinical events were represented at input. Unlike PORTER, where composite tokens served only as output labels during pretraining, Fixed-Vocab FM used the same composite tokens as both input identifiers and output labels. Its input layer was therefore a learned embedding table indexed by composite token, and its prediction head predicted the next composite token from the finite pretraining vocabulary. Pretraining data, backbone transformer, temporal encoding, and training schedule were otherwise matched to PORTER. During transfer evaluation, Fixed-Vocab FM could only consume composite tokens observed during SickKids pretraining. Composite tokens outside this vocabulary were dropped before representation extraction. The patient-level text serialization comparator followed a representative recent approach.[15] For each task, the patient timeline up to the task-specific prediction time was serialized into a structured markdown document containing demographics, recent body metrics, recent vital signs, recent laboratory results, past visits, and visit-level summaries (see Supplementary Figure S4 for an illustrative serialized patient timeline). Recent measurements were drawn from curated code lists and limited to the most recent available values. As in the prior study, each document was truncated to 8,192 input tokens after prepending a task-specific retrieval instruction (see Supplementary Table S5 for the full set of task-specific instructions). The resulting text was encoded with Qwen3-Embedding-8B using last-token pooling to produce a single 4,096-dimensional task-conditioned patient representation. Because the retrieval instruction was task-specific, patient-level embeddings were generated separately for each task. In contrast, PORTER and Fixed-Vocab FM produced task-agnostic patient representations that were reused across downstream tasks. We therefore used the patient-level text serialization comparator only in in-domain evaluation, not in cross-vocabulary or cross-site evaluations. 2.6
Ablation Experiments
We conducted two sets of ablation experiments to evaluate key architectural decisions, with all variants pretrained on the same SickKids training split using the same backbone, pretraining objective, optimizer, and training schedule. 2.6.1
Text Encoder Ablation
We compared three frozen text encoders: BioLORD-2023 (768-dimensional), BGE-M3 (1,024-dimensional), and Qwen3-Embedding-8B (4,096-dimensional), plus a non-semantic random-embedding baseline. For the random baseline, each unique rendered event-description string was assigned a fixed 4,096-dimensional vector drawn from a standard normal distribution and L2-normalized to unit length. Vectors were deterministically keyed by the rendered event-description string, so identical event-description strings received identical vectors across vocabularies, whereas semantically equivalent but textually distinct event-description strings received uncorrelated vectors. This baseline preserves event-description identity while removing semantic structure. The lookup table was cached in the same manner as the text-encoder embeddings. Because embedding dimensionality differed across encoders, the input dimension of the text projection MLP (and consequently its parameter count) also varied, while the templated event descriptions and numeric pathway remained identical. 2.6.2
Numeric Encoding Ablation
We compared three architectural strategies for integrating numeric values into the language-grounded event representation: 1) PORTER, which encodes numeric values through the numeric pathway and integrates them with the text-derived event representation through FiLM; 2) PORTER-NoNum, which removes the numeric pathway and FiLM entirely and uses only the text-derived representation; and 3) PORTER-NumText, which removes the numeric pathway and FiLM but incorporates numeric values, units, and reference ranges directly into the templated event descriptions before encoding, testing whether the frozen text encoder alone could represent numeric magnitude when numeric information was expressed in language.
7
Portable EHR Representations
2.7
Downstream Evaluation
2.7.1
Full-Shot Evaluation
Preprint
After pretraining, the backbone was frozen and hidden states were extracted at prediction time for each of the 74 downstream binary classification tasks. Linear probes were trained using L2-regularized logistic regression implemented in Scikit-learn[40], with features standardized to zero mean and unit variance using statistics computed on the training subset of each task's split. Models were fit with LBFGS for up to 10,000 iterations, with inverse regularization strength selected from {1, 10-1 , 10-2 , 10-3 , 10-4 } on the validation set. Performance was reported as AUROC on the held-out temporal test split. 2.7.2
Sample Efficiency Evaluation
To assess sample efficiency, models were evaluated across labeled training set sizes (k) ranging from 2 to 32,768 examples, in powers of two. For each shot size, 10 independent iterations were conducted using a fixed sequence of random seeds applied identically across tasks. In each iteration, k training examples were drawn from the task-specific training set using balanced sampling (equal positive and negative examples where possible), while the validation and test sets remained unchanged. For tasks with fewer than k/2 positive instances available, all available positive instances were included, with the remaining examples drawn from the negative class. The same linear probe protocol was used. Task performance was reported as the mean across the 10 iterations for each shot size. 2.7.3
Cross-Vocabulary Evaluation
To isolate vocabulary shift from differences in patient population, outcome definitions, and temporal evaluation period, the cross-vocabulary evaluation used the same underlying SickKids admissions, prediction times, labels, and downstream splits as the in-domain evaluation. Only the source naming system used to render event descriptions was changed. In the primary PORTER setup, OMOP-derived event descriptions were constructed from OMOP concept names. In the cross-vocabulary setup, native SEDAR-derived event descriptions for the same patient timelines were regenerated using SEDAR concept names. For PORTER, the cached text-embedding table was regenerated from these alternate event descriptions using the same frozen text encoder, while the projection layer, FiLM module, and temporal backbone remained unchanged and frozen. This tested whether a PORTER model pretrained with OMOP-derived descriptions could consume the same clinical timelines rendered using a source naming system not seen during pretraining, without retraining the input pathway or backbone. Fixed-Vocab FM could not be evaluated in this setting because its input embedding table was indexed by composite event tokens observed during pretraining, and SEDAR composite event tokens had no learned input embeddings. Downstream evaluation followed the same linear-probe full-shot and sample-efficiency evaluation protocols used for in-domain evaluation. To provide an upper-bound reference, we trained a Fixed-Vocab FM from scratch on SickKids data using composite event tokens constructed from native SEDAR event-attribute combinations, and evaluated it under the same full-shot protocol. 2.7.4
Cross-Site Evaluation
Cross-site generalization was evaluated by applying models pretrained on SickKids to MIMIC. For PORTER, MIMIC OMOP-derived event descriptions were rendered using the same templates, with slots populated from MIMIC OMOP concept names, then encoded with the same frozen text encoder to construct a MIMIC-specific cached embedding table. The SickKids-trained projection layer, FiLM module, and temporal backbone were reused unchanged. Thus, PORTER adapted its input cache to the MIMIC OMOP-derived description set through text encoding, without modifying learned model weights before downstream evaluation. Fixed-Vocab FM was transferred by applying its SickKids-trained composite-token vocabulary to MIMIC composite tokens. MIMIC events whose composite token was not observed during SickKids pretraining were dropped before representation extraction. We quantified event-drop rates as the fraction of MIMIC events removed by this procedure. In contrast, PORTER does not drop any events: any MIMIC event whose event description the frozen text encoder can read produces a representation. Downstream evaluation for PORTER and Fixed-Vocab FM followed the same linear-probe full-shot and sample-efficiency evaluation protocols used for in-domain evaluation.
8
Portable EHR Representations
Preprint
To provide an upper-bound reference, we trained a Fixed-Vocab FM from scratch on MIMIC data using the same pretraining procedure and evaluated it under the full-shot protocol. 2.8
Mechanistic Analyses
We performed representation-level analyses to characterize mechanisms underlying cross-vocabulary transfer and numeric encoding. To assess whether cross-vocabulary transfer preserved patient-level geometry, we extracted test-set patient representations from PORTER variants with different text encoders after rendering the same patient timelines under two source naming systems: OMOP-derived descriptions and native SEDARderived descriptions. The conditions differed only in the names used to construct event descriptions, whereas the underlying events, timestamps, and numeric metadata were identical. We computed pairwise cosine distances among test patients under each condition. Relative geometry was quantified as the Spearman correlation between the OMOP-derived and SEDAR-derived distance matrices. Preservation of distance scale was summarized by ordinary least-squares regression of SEDAR-derived distances on OMOP-derived distances, with slope near 1 and intercept near 0 indicating minimal compression, expansion, or shift across naming systems. At the event level, we evaluated synonym invariance and numeric sensitivity of the event representations passed to the transformer for PORTER, PORTER-NoNum, and PORTER-NumText. Synonym invariance was assessed using OMOP-derived and SEDAR-derived event-description pairs across drug, measurement, procedure, condition, and observation events. Because multiple SEDAR concept names could correspond to the same OMOP concept, this analysis tested whether SEDAR-derived naming variants were mapped near their shared OMOP-derived reference. We retained concepts with at least two distinct SEDAR-derived event descriptions and excluded queries in which the OMOP concept name appeared within the SEDAR-derived description. For each concept, up to 10 SEDAR-derived descriptions were used as queries. The matched OMOP-derived description served as the within-concept reference, and 32 sampled OMOP-derived descriptions from other concepts in the same domain served as distractors. Cosine similarity was compared between query-reference and query-distractor pairs. Numeric sensitivity was assessed among measurement concepts with at least 1,000 numeric values recorded in a common unit. For each concept, we constructed the event representation passed to the transformer at empirical deciles of the observed value distribution for PORTER, PORTER-NoNum, and PORTER-NumText. Representations were L2-normalized. For each concept, cosine distance from the first-decile representation to each subsequent decile representation was computed, and Spearman correlation between decile rank and distance was measured. Higher positive correlation indicated stronger monotonic sensitivity of the representation to numeric magnitude. PORTER-NoNum was invariant to decile by construction, so its correlation was undefined. 2.9
Compute Cost Analysis
Total compute for PORTER was estimated as the sum of backbone pretraining floating-point operations (FLOPs), one-time event description text-embedding FLOPs, and backbone extraction FLOPs for generating patient representations over the downstream task cohorts. Backbone pretraining was estimated using the standard 6ND[41] approximation, where N is the number of non-embedding backbone parameters and D is the number of EHR event tokens processed during training. Event description text embeddings were computed once over unique event descriptions and estimated as 2Ntext Dtext forward-pass FLOPs through BioLORD, where Ntext is the number of non-embedding text-encoder parameters and Dtext is the total number of event-description tokens. Backbone extraction was estimated as the forward-pass FLOPs of the pretrained EHR backbone over patient records in the task cohort. For Fixed-Vocab FM, only backbone pretraining and backbone extraction FLOPs applied. For the text serialization comparator, per-task compute was estimated as Qwen3-Embedding-8B forward-pass FLOPs summed over all patient records in that task’s cohort. Because the serialized input and retrieval instruction were task-specific, representations could not be shared across tasks, so total compute scaled with the number of downstream tasks. We report aggregate compute across the 74 SickKids tasks and its effective scaling factor relative to PORTER. 2.10
Statistical Analysis
Model comparisons were based on paired per-task AUROC values. For each model pair, we computed per-task AUROC differences across the 74 SickKids tasks for in-domain and cross-vocabulary evaluations, or the 36 9
Portable EHR Representations
Preprint
MIMIC tasks for cross-site evaluation, and tested whether the paired differences were symmetrically centered around zero using a two-sided Wilcoxon signed-rank test[42] implemented in SciPy.[43] When multiple comparisons were performed within an ablation family (text encoder choice or numeric encoding strategy), p-values were adjusted using Holm's step-down procedure [44] to control the family-wise error rate at α = 0.05. Holm correction was applied separately within each evaluation setting: in-domain, cross-vocabulary, and cross-site.
3
RESULTS
3.1
Study Cohorts, Vocabularies, and Pretraining Compute
The SickKids and MIMIC pretraining cohorts comprised 2,100,646 and 339,989 patients, with 220.3 and 181.3 million clinical events, respectively (Table 1). Downstream evaluation included 101,404 SickKids admissions and 58,513 MIMIC admissions. The SickKids OMOP, SEDAR, and MIMIC OMOP source concept sets included 15,516, 105,551, and 32,032 unique clinical concepts, with 19,686, 523,724, and 36,524 unique rendered event-descriptions, respectively. SickKids OMOP and MIMIC OMOP shared 1,153 event-descriptions, whereas SickKids OMOP and SEDAR shared only 2. Across models, pretraining compute was similar at approximately 1.0 × 1018 FLOPs. Trainable input-pathway parameters ranged from 1.18M for PORTER variants without numeric integration to 75.5M for the Fixed-Vocab FM learned vocabulary embedding table (Supplementary Table S6). Table 1: Characteristics of pretraining and task cohorts. Characteristic
SickKids
MIMIC
2,100,646 220,296,987 1 (0–256) 49.7
339,989 181,321,218 17 (2–553) 53.5
58,426 101,404 7 (2–13) 2 (1–6) 45.9
44,055 58,513 54 (34–70) 4 (2–7) 61.9
a
Pretraining Cohort Patients, n Clinical events, n Timeline duration in days, median (IQR) Female sex, % Task Cohort b Patients, n Admissions, n Age at admission, median (IQR) Length of stay in days, median (IQR) Female sex, % a
Pretraining cohorts include patients and events used for self-supervised pretraining of the SickKids and the reference MIMIC foundation models. b Task cohorts include admissions considered for downstream clinical prediction evaluation. The MIMIC cohort was also used for the external evaluation of the SickKids foundation models. Abbreviations: IQR, interquartile range; SickKids, The Hospital for Sick Children; MIMIC, Medical Information Mart for Intensive Care.
3.2
PORTER Matches Fixed-Vocab FM In-Domain and Improves Transfer Across Vocabularies and Institutions
On the 74 SickKids in-domain tasks, PORTER and Fixed-Vocab FM did not differ in mean AUROC (0.884 vs 0.884, p=0.942; Table 2, Supplementary Table S7). In the cross-vocabulary setting, SickKids timelines were re-expressed using SEDAR-derived event descriptions. PORTER achieved a mean AUROC of 0.848 without retraining or vocabulary mapping, recovering 97.1% of the performance of an upper-bound reference model trained directly on SEDAR composite event tokens (mean AUROC, 0.873). Fixed-Vocab FM could not be directly evaluated in this setting because its learned input vocabulary did not contain SEDAR composite event tokens. On cross-site evaluation in MIMIC, PORTER had higher AUROC than Fixed-Vocab FM on 31 of 36 tasks (mean AUROC 0.823 vs 0.810, p<0.001). For Fixed-Vocab FM, 69% of MIMIC events contained unseen composite tokens and were dropped during representation extraction, whereas PORTER produced representations for all MIMIC events. An upper-bound reference model trained directly on MIMIC achieved mean AUROC of 0.848. Across labeled training-set sizes, PORTER and Fixed-Vocab FM were
10
Portable EHR Representations
Preprint
similar in-domain at larger sample sizes, while PORTER was higher in few-shot in-domain settings and across sample sizes in MIMIC (Supplementary Figure S5). Table 2: In-domain, cross-vocabulary, and cross-site mean AUROC of PORTER and Fixed-Vocab FM.
PORTER Fixed-Vocab FM
In-domain (SK)
Cross-vocabulary (SK)
Cross-site (MIMIC)
0.884 0.884
0.848 –a
0.823 0.810***
Bold: the statistical reference model. p < 0.001 vs PORTER (reference) using two-sided Wilcoxon signed-rank test. a Fixed-Vocab FM was not evaluated in cross-vocabulary transfer because SEDAR composite event tokens had no learned input embeddings. Abbreviations: AUROC, area under the receiver operating characteristic curve; PORTER, Portable EHR Representations; EHR, electronic health records; FM, foundation model; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise-wide Data in Azure Repository.
***
3.3
PORTER Outperforms a Task-Specific Text Serialization Comparator on AUROC, Sample Efficiency, and Amortized Compute
PORTER had higher full-shot AUROC than the patient-level text serialization comparator on 69 of 74 in-domain tasks (mean AUROC 0.884 vs 0.863, p<0.001; Figure 2A, Supplementary Table S8). Across labeled training-set sizes, PORTER had higher AUROC from 4 examples per task onward, with larger differences between 32 to 512 examples per task (Figure 2B). The text serialization comparator required less compute when only a small number of patient-task representations were generated, whereas PORTER required less compute once representations were generated for more than approximately 256 admissions per task, and required 329-fold fewer amortized FLOPs across all 74 task cohorts (Figure 2C). 3.4
Text Encoder Choice Determines Cross-Vocabulary Transfer More Than In-Domain Performance
Holding the numeric pathway and FiLM fixed, encoder-related AUROC differences were largest in the cross-vocabulary setting, where SickKids OMOP and SEDAR shared almost no event-description strings (Table 3, Supplementary Table S9). In-domain differences were statistically significant but small: 0.884 for BioLORD, 0.881 for Qwen3, 0.881 for BGE-M3, and 0.879 for Random (all p<0.001 vs BioLORD). Crossvocabulary differences were larger: 0.848, 0.840, 0.818, and 0.780, respectively, with the largest degradation observed for Random (all p<0.001 vs BioLORD). On cross-site evaluation, Qwen3 (0.825) and BGE-M3 (0.821) were statistically indistinguishable from BioLORD (0.823), while Random was lower (0.812; p<0.001). Sample-efficiency curves showed limited in-domain differences but larger cross-vocabulary and cross-site performance deficits relative to BioLORD across labeled sample sizes, again most prominently for Random (Supplementary Figure S6). Table 3: Effect of text encoder choice on in-domain, cross-vocabulary, and cross-site mean AUROC. Input representation
Text Encoder Parameters In-domain (SK) Cross-vocabulary (SK) Cross-site (MIMIC)
BioLORD Qwen3 BGE-M3 Random Embeddings
0.1B 8B 0.6B –
0.884 0.881*** 0.881*** 0.879***
0.848 0.840*** 0.818*** 0.780***
0.823 0.825 0.821 0.812***
Bold: the statistical reference model. Significance vs BioLORD (reference); *** p < 0.001 (two-sided Wilcoxon signed-rank test, Holm-corrected across 3 reference comparisons per setting). Abbreviations: AUROC, area under the receiver operating characteristic curve; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care; B, billion.
Preservation of patient-level representation geometry between OMOP-derived and SEDAR-derived event descriptions followed the same across-encoder ranking as cross-vocabulary AUROC. Pairwise cosine distances 11
Portable EHR Representations
Preprint
Figure 2: Performance, sample efficiency, and compute for PORTER versus the patient-level text serialization comparator. (A) Per-task AUROC difference between PORTER and the patient-level text serialization comparator (Qwen3-Embedding-8B with patient-level text serialization) across 74 SickKids clinical prediction tasks evaluated in-domain. Each point represents one task. Blue points indicate tasks where PORTER performed better, and red points indicate tasks where the text serialization comparator performed better. Tasks are grouped by clinical category, with per-category win counts annotated. (B) Mean AUROC difference between PORTER and the text serialization comparator, left axis, and mean AUROC for each approach, right axis, as a function of labeled training examples per task. Shaded region indicates ±1 standard error of the mean difference across tasks. (C) Percentage reduction in total floating-point operations achieved by PORTER relative to the text serialization comparator, left axis, alongside total FLOPs for each approach, right axis. PORTER patient representations are computed once and reused across tasks, whereas the text serialization comparator requires a separate forward pass through Qwen3-Embedding-8B for each patient-task pair. As a result, comparator compute scales with the number of downstream tasks, while PORTER’s amortized compute advantage increases with downstream reuse. Abbreviations: PORTER, Portable EHR Representations; EHR, electronic health records; AUROC, area under the receiver operating characteristic curve; FLOPs, floating-point operations. between test patients computed under the two description sources had Spearman correlations of 0.613 for BioLORD, 0.606 for Qwen3, 0.515 for BGE-M3, and 0.267 for Random. Distance-scale preservation showed the same ranking, with OLS slopes/intercepts of 0.660/0.018, 0.602/0.086, 0.478/0.137, and 0.318/0.210, respectively (Supplementary Figure S7).
12
Portable EHR Representations
3.5
Preprint
FiLM Numeric Pathway Improves Numeric Sensitivity Without Disrupting Concept Identity
Holding the text encoder fixed, mean AUROC differed by numeric encoding strategy (Table 4, Supplementary Table S10). PORTER had the highest mean AUROC in all three settings: 0.884 in-domain versus 0.880 for PORTER-NumText and 0.873 for PORTER-NoNum; 0.848 cross-vocabulary versus 0.839 and 0.837; and 0.823 cross-site versus 0.801 and 0.804, respectively (all p<0.001 vs PORTER). Sample-efficiency curves followed the same overall pattern (Supplementary Figure S8). Table 4: Effect of numeric encoding strategy on in-domain, cross-vocabulary, and cross-site mean AUROC. Variant PORTER PORTER-NoNum PORTER-NumText
In-domain (SK)
Cross-vocabulary (SK)
Cross-site (MIMIC)
0.884 0.873*** 0.880***
0.848 0.837*** 0.839***
0.823 0.804*** 0.801***
Bold: the statistical reference model. Significance vs PORTER (reference); *** p < 0.001 (two-sided Wilcoxon signed-rank test, Holm-corrected across 2 reference comparisons per setting). Abbreviations: AUROC, area under the receiver operating characteristic curve; PORTER, Portable EHR Representations; EHR, electronic health records; NoNum, no numeric pathway (text-derived representations only); NumText, numeric values rendered as text; SK, SickKids; MIMIC, Medical Information Mart for Intensive Care. MIMIC cross-site results include 36 tasks.
Analyses of the event input representations passed to the transformer showed that all three variants preserved clinical concept identity, with within-concept cosine similarity exceeding across-concept similarity across event domains (Supplementary Figure S9). For numeric sensitivity, the Spearman correlation for each measurement concept between value-decile rank and cosine distance from the first-decile representation had median 1.00 for PORTER and 0.67 for PORTER-NumText. PORTER-NoNum was invariant to numeric value by construction, so this correlation was undefined.
4
DISCUSSION
PORTER enabled direct cross-vocabulary transfer to patient timelines expressed with event descriptions not used during pretraining, achieving 97.1% of the AUROC of a reference model trained directly on the target vocabulary. This transfer required neither PORTER retraining nor vocabulary mapping. PORTER also matched a fixed-vocabulary foundation model across 74 in-domain tasks and improved transfer on 31 of 36 tasks at an external site. Compared with a task-specific patient-level text serialization comparator, PORTER achieved higher AUROC on 69 of 74 in-domain tasks and showed an amortized compute advantage that increased with the number of downstream tasks and patients, reaching 329-fold in the full evaluation. Ablations clarified the distinct roles of PORTER’s architectural components. The temporal backbone preserved in-domain performance across input representations. Cross-vocabulary performance tracked how well each encoder preserved patient-level representation geometry across vocabularies, an emergent property of the pretrained representations that is not explained by encoder scale alone. The dedicated numeric pathway improved sensitivity to numeric magnitude without degrading clinical concept identity. PORTER shows that vocabulary-independent inputs can support self-supervised EHR foundation model pretraining without compromising task-agnostic representation learning. Prior work has shown that encoding clinical concepts or events through text can reduce dependence on fixed vocabularies and improve transfer across institutions and languages.[18, 19, 22, 23] However, these approaches generally use text-based representations within supervised settings. Even when self-supervised objectives are used, the resulting model is typically adapted through downstream fine-tuning rather than used as a frozen source of task-agnostic patient representations. PORTER instead uses autoregressive pretraining to learn a temporal backbone that is frozen and reused across downstream tasks through linear probes. The cross-vocabulary evaluation isolates vocabulary shift from the population and practice differences that confound cross-site comparisons. This setting also reflects a common deployment challenge in which models trained on retrospective research datasets may later be applied to production feeds that use different vocabularies or naming conventions. Fixed-vocabulary foundation models lack a direct mechanism for this setting without vocabulary mapping or
13
Portable EHR Representations
Preprint
retraining. Together with the in-domain parity result, these findings support vocabulary-independent input representations as a viable design choice for structured EHR foundation models. Ablation experiments clarified the distinct roles of PORTER’s architectural components. Under in-domain evaluation, random embeddings preserved much of PORTER’s AUROC. This indicates that the temporal backbone can learn useful clinical dynamics when clinical events are represented consistently, even without semantic structure in the input vectors. In cross-vocabulary evaluation, however, random embeddings degraded performance substantially, likely because distinct descriptions of the same clinical concept were assigned unrelated embeddings. Across PORTER variants using different text encoders, cross-vocabulary AUROC tracked the extent to which each encoder preserved patient-level pairwise distance structure across vocabularies. Next-event prediction does not optimize for this geometry, so this correspondence is an emergent property of the pretrained representations and points to semantic alignment as the basis for vocabulary transfer. Finally, using the dedicated numeric pathway increased ordinal sensitivity from 0.67 to 1.00 compared to text-encoded numeric values while preserving within-concept representation structure. This indicates that FiLM-based numeric integration strengthened numeric sensitivity in the event input representation without disrupting the concept semantics supplied by the event-description embedding. The advantage of the dedicated numeric pathway over rendering numeric values as text widened in cross-vocabulary and crosssite evaluations, consistent with normalized numeric features transferring more reliably than text-rendered magnitudes. In PORTER, this normalization is reference-range anchored when available and uses a bounded log-magnitude fallback otherwise. Together, these findings indicate that PORTER's components serve distinct and complementary roles and can in principle be improved independently. The encoder ablation findings suggest that text-encoder scale is not the primary determinant of PORTER’s performance. Across evaluations, BioLORD, a 109M-parameter biomedical encoder trained to align biomedical concept names with ontology-informed definitions, performed on par with or better than general-purpose encoders up to 63 times larger, with the largest advantage in cross-vocabulary evaluation. This pattern may reflect PORTER’s narrower requirement for the text encoder, which is to embed short clinical event descriptions so that alternative descriptions of the same clinical concepts remain close across vocabularies. By contrast, Qwen3-Embedding and BGE-M3 are general-purpose embedding models optimized across broad multilingual, retrieval, relevance-ranking, and long-context settings. Standard embedding benchmarks evaluate many useful capabilities[45], but they do not directly measure the properties most relevant to language-grounded EHR event modeling, including preservation of patient-level geometry across vocabularies and ordinal sensitivity to numeric values. Developing encoder benchmarks around these properties could make encoder selection more efficient and may improve cross-vocabulary transfer without changing the temporal backbone or numeric pathway. The text encoder findings also inform the comparison between PORTER and patient-level text serialization. In PORTER, language grounding occurs at the event level. Each clinical event is represented using a frozen text encoder and dedicated numeric pathway, and the temporal backbone integrates sequences of event input representations into reusable patient representations. Patient-level serialization instead represents the full patient timeline as text and encodes it directly as a patient-level representation, often within a task-specific prompt. This provides flexible task conditioning but requires the model to represent the full task-conditioned patient timeline. Prior patient-level serialization work suggests that this setting favors larger general-purpose encoders.[15] The compute profiles also differ. PORTER’s event-description embeddings are computed once per unique description, and the resulting patient representations can be reused across downstream tasks. In contrast, the task-specific serialization comparator we evaluate requires a languagemodel forward pass for each patient-task pair. Recent rubric-based serialization methods synthesize an extraction schema from labeled examples and report higher accuracy than direct serialization[14], although they still build patient representations tied to a specific task. These differences suggest that PORTER suits high-throughput, population-scale prediction, whereas patient-level serialization may suit settings that value flexible, task-specific conditioning over reuse. Several limitations should be noted. All models in the main experiments were pretrained on data from a single pediatric hospital, and cross-site evaluation was limited to one external adult intensive care setting, where vocabulary mismatch and population shift co-occur. All downstream evaluations used linear probes on frozen representations for binary classification tasks. Performance under fine-tuning, more expressive adaptation methods[46, 47], regression tasks, zero-shot prediction, or generative settings has not been assessed. We also kept the text encoder frozen and did not test fine-tuning it end-to-end during pretraining, which could change which encoder properties matter.[23] PORTER depends on the availability and quality of event descriptions and numeric metadata, including reference ranges, which may vary across institutions and data pipelines. Finally, although PORTER removes fixed-vocabulary dependence on the input side,
14
Portable EHR Representations
Preprint
its autoregressive pretraining objective still uses a fixed output vocabulary for next-event prediction. The language-grounded input pathway is independent of this objective and could be paired with alternatives such as time-to-event or continuous-time generative formulations.[5–7, 48] Because these alternatives are also typically defined over a fixed set of output codes, reducing output-side vocabulary dependence remains future work. In conclusion, PORTER shows that language-grounded event representations can make structured EHR foundation models portable across vocabularies while preserving in-domain performance and allowing patient representations to be reused efficiently across downstream tasks. By modeling concept semantics, numeric values, and temporal dynamics separately, PORTER reduces dependence on fixed vocabularies and provides a practical route toward EHR foundation models that can generalize across deployment settings and institutions.
5
Data Availability
The SickKids dataset cannot be made publicly available due to patient privacy restrictions. Relevant data are available upon reasonable request to the corresponding author. The MIMIC dataset is publicly available through PhysioNet (https://physionet.org/content/mimiciv/1.0/) subject to credentialing and a data use agreement.
6
Code Availability
The codebase for EHR foundation model training will be made publicly available at https://github.com/ sungresearch/ehr-fm.
7
Acknowledgements
LS is supported by the Canada Research Chair in Pediatric Oncology Supportive Care. We thank Jason Alan Fries and Natalie Pageler for their helpful feedback on an earlier draft of this manuscript.
8
Funding
This research did not receive funding.
9
Author Contribution
L.L.G. conceptualized and designed the study with input from all authors. L.L.G. performed all experiments, analyzed and interpreted results with input from all authors. L.L.G. wrote the manuscript with input from all authors. L.L.G, A.P.Y, E.V, and L.S read and approved the final manuscript.
10
Competing Interests
The authors declare no competing interests.
References [1] Ethan Steinberg, Ken Jung, Jason A. Fries, Conor K. Corbin, Stephen R. Pfohl, and Nigam H. Shah. Language models are an effective representation learning technique for electronic health record data. Journal of Biomedical Informatics, 113:103637, 2021. ISSN 1532-0464. doi: 10.1016/j.jbi.2020.103637. URL https://www.sciencedirect.com/science/article/pii/S1532046420302653. [2] Pawel Renc, Yugang Jia, Anthony E. Samir, Jaroslaw Was, Quanzheng Li, David W. Bates, and Arkadiusz Sitek. Zero shot health trajectory prediction using transformer. npj Digital Medicine, 7(1):256, 2024. ISSN 2398-6352. doi: 10.1038/s41746-024-01235-0. URL https://doi.org/10.1038/s41746-024-01235-0. [3] Shane Waxler, Paul Blazek, Davis White, Daniel Sneider, Kevin Chung, Mani Nagarathnam, Patrick Williams, Hank Voeller, Karen Wong, Matthew Swanhorst, Sheng Zhang, Naoto Usuyama, Cliff Wong,
15
Portable EHR Representations
Preprint
Tristan Naumann, Hoifung Poon, Andrew Loza, Daniella Meeker, Seth Hain, and Rahul Shah. Generative medical event models improve with scale. arXiv, 2508.12104, 2025. URL https://arxiv.org/abs/ 2508.12104. [4] Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digital Medicine, 4(1):86, 2021. ISSN 2398-6352. doi: 10.1038/s41746-021-00455-y. URL https://doi.org/10.1038/ s41746-021-00455-y. [5] Ethan Steinberg, Jason Fries, Yizhe Xu, and Nigam Shah. MOTOR: A time-to-event foundation model for structured medical records. International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=NialiwI2V6. [6] Artem Shmatko, Alexander Wolfgang Jung, Kumar Gaurav, Søren Brunak, Laust Hvas Mortensen, Ewan Birney, Tom Fitzgerald, and Moritz Gerstung. Learning the natural history of human disease with generative transformers. Nature, 647(8088):248–256, 2025. ISSN 1476-4687. doi: 10.1038/ s41586-025-09529-3. URL https://doi.org/10.1038/s41586-025-09529-3. [7] Charles Gadd, Krishna Gokhale, Aditya Acharya, Jennifer Cooper, Francesca Crowe, Leah Fitzsimmons, Thomas Jackson, Krishnarajah Nirantharakumar, Christopher Yau, Rebecca Birch, Marco Canducci, Dominic Danks, Alexander d’Elia, Alastair Denniston, Sarah Flanagan, Suzy Gallier, Naijie Guan, Xin Guan, Imane Guellil, Georgios Gkoutos, Shamil Haroon, Eleanor Hathaway, Louise Jackson, Janet Lord, Zeinab Majid, Tom Marshall, George Morris, Charlotte Owen, Elizabeth Sapey, Chris Sainsbury, Charlotte Spurway, Peter Tino, Steven Wambua, Amaya Azcoaga-Lorenzo, Colin McCowan, Luciana Rocha Pedro, Muhammad Usman, Natalia Hong, Sara Matijevic, Kaspar Martens, Tim Williams, Puja Myles, and The OPTIMAL Consortium. SurvivEHR: a competing risks, time-to-event foundation model for multiple long-term conditions from primary care electronic health records. npj Digital Medicine, 2026. ISSN 2398-6352. doi: 10.1038/s41746-026-02709-z. URL https://doi.org/10.1038/ s41746-026-02709-z. [8] Chao Pang, Jiheum Park, Xinzhuo Jiang, Nishanth Parameshwar Pavinkurve, Krishna S. Kalluri, Shalmali Joshi, Noémie Elhadad, and Karthik Natarajan. CEHR-XGPT: A scalable multi-task foundation model for electronic health records. arXiv, 2509.03643, 2025. URL https://arxiv.org/abs/2509.03643. [9] Lin Lawrence Guo, Ethan Steinberg, Scott Lanyon Fleming, Jose Posada, Joshua Lemmon, Stephen R. Pfohl, Nigam Shah, Jason Fries, and Lillian Sung. EHR foundation models improve robustness in the presence of temporal distribution shift. Scientific Reports, 13(1):3767, 2023. ISSN 2045-2322. doi: 10.1038/s41598-023-30820-8. URL https://doi.org/10.1038/s41598-023-30820-8. [10] Joshua Lemmon, Lin Lawrence Guo, Ethan Steinberg, Keith E Morse, Scott Lanyon Fleming, Catherine Aftandilian, Stephen R Pfohl, Jose D Posada, Nigam Shah, Jason Fries, and Lillian Sung. Self-supervised machine learning using adult inpatient data produces effective models for pediatric clinical prediction tasks. Journal of the American Medical Informatics Association, 30(12):2004–2011, 2023. ISSN 1527-974X. doi: 10.1093/jamia/ocad175. URL https://doi.org/10.1093/jamia/ocad175. [11] Lin Lawrence Guo, Jason Fries, Ethan Steinberg, Scott Lanyon Fleming, Keith Morse, Catherine Aftandilian, Jose Posada, Nigam Shah, and Lillian Sung. A multi-center study on the adaptability of a shared foundation model for electronic health records. npj Digital Medicine, 7(1):171, 2024. ISSN 2398-6352. doi: 10.1038/s41746-024-01166-w. URL https://doi.org/10.1038/s41746-024-01166-w. [12] Lin Lawrence Guo, Santiago Eduardo Arciniegas, Adam Paul Yan, Jason Fries, George A Tomlinson, and Lillian Sung. Systematic review of foundation models for structured electronic health records. Journal of the American Medical Informatics Association, 33(6):1190–1198, 2026. ISSN 1527-974X. doi: 10.1093/jamia/ocag033. URL https://doi.org/10.1093/jamia/ocag033. [13] Lin Lawrence Guo, Santiago Eduardo Arciniegas, Joseph Jihyung Lee, Adam Paul Yan, George Tomlinson, Jason Fries, and Lillian Sung. Tokenization tradeoffs in structured EHR foundation models. arXiv, 2603.15644, 2026. URL https://arxiv.org/abs/2603.15644. [14] Ilker Demirel, Lawrence Shi, Zeshan Hussain, and David Sontag. LLMs can construct powerful representations and streamline sample-efficient supervised learning. arXiv, 2603.11679, 2026. URL https://arxiv.org/abs/2603.11679. [15] Stefan Hegselmann, Georg von Arnim, Tillmann Rheude, Noel Kronenberg, David A. Sontag, Gerhard Hindricks, Roland Eils, and Benjamin Wild. Large language models are powerful electronic health record encoders. arXiv, 2502.17403, 2025. URL https://arxiv.org/abs/2502.17403.
16
Portable EHR Representations
Preprint
[16] Jifan Gao, Mahmudur Rahman, John Caskey, Madeline Oguss, Ann O’Rourke, Randall Brown, Anne Stey, Anoop Mayampurath, Matthew M. Churpek, Guanhua Chen, and Majid Afshar. MoMA: a mixtureof-multimodal-agents architecture for enhancing clinical prediction modelling. npj Digital Medicine, 9(1):46, 2026. ISSN 2398-6352. doi: 10.1038/s41746-025-02219-4. URL https://doi.org/10.1038/ s41746-025-02219-4. [17] Hejie Cui, Alyssa Unell, Bowen Chen, Jason Alan Fries, Emily Alsentzer, Sanmi Koyejo, and Nigam H. Shah. TIMER: temporal instruction modeling and evaluation for longitudinal clinical records. npj Digital Medicine, 8(1):577, 2025. ISSN 2398-6352. doi: 10.1038/s41746-025-01965-9. URL https: //doi.org/10.1038/s41746-025-01965-9. [18] Matthias Kirchler, Matteo Ferro, Veronica Lorenzini, Robin P. van de Water, Christoph Lippert, Andrea Ganna, and FinnGen. Large language models improve transferability of electronic health record-based predictions across countries and coding systems. npj Digital Medicine, 9(1):177, 2026. ISSN 2398-6352. doi: 10.1038/s41746-026-02363-5. URL https://doi.org/10.1038/s41746-026-02363-5. [19] Kyunghoon Hur, Jiyoung Lee, Jungwoo Oh, Wesley Price, Younghak Kim, and Edward Choi. Unifying heterogeneous electronic health records systems via text-based code embedding. In Proceedings of the Conference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pages 183–203. PMLR, 2022. URL https://proceedings.mlr.press/v174/hur22a.html. [20] Doudou Zhou, Han Tong, Linshanshan Wang, Suqi Liu, Xin Xiong, Ziming Gan, Romain Griffier, Boris P. Hejblum, Yun-Chung Liu, Chuan Hong, Clara-Lea Bonzel, Tianrun Cai, Kevin Pan, YukLam Ho, Lauren Costa, Vidul A. Panickan, J. Michael Gaziano, Kenneth D. Mandl, Vianney Jouhet, Rodolphe Thiebaut, Zongqi Xia, Kelly Cho, Katherine Liao, and Tianxi Cai. Representation learning to advance multi-institutional studies with electronic health record data from us and france. Nature Communications, 17(1):4800, 2026. ISSN 2041-1723. doi: 10.1038/s41467-026-71152-1. URL https: //doi.org/10.1038/s41467-026-71152-1. [21] Rafi Al Attrach, Rajna Fani, David Restrepo, Yugang Jia, and Peter Schüffler. Rethinking tokenization for clinical time series: When less is more. arXiv, 2512.05217, 2025. URL https://arxiv.org/abs/ 2512.05217. [22] Kyunghoon Hur, Jungwoo Oh, Junu Kim, Jiyoun Kim, Min Jae Lee, Eunbyeol Cho, Seong-Eun Moon, Young-Hak Kim, Louis Atallah, and Edward Choi. GenHPF: General healthcare predictive framework for multi-task multi-source learning. IEEE Journal of Biomedical and Health Informatics, 28(1):502–513, 2024. doi: 10.1109/JBHI.2023.3327951. URL https://doi.org/10.1109/JBHI.2023.3327951. [23] Kyunghoon Hur, Heeyoung Kwak, Jinsu Jang, Nakhwan Kim, and Edward Choi. Multi-lingual multiinstitutional electronic health record based predictive model. arXiv, 2604.00027, 2026. doi: 10.48550/ arXiv.2604.00027. URL https://arxiv.org/abs/2604.00027. [24] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. FiLM: Visual reasoning with a general conditioning layer. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. doi: 10.1609/aaai.v32i1.11671. URL https://doi.org/10.1609/aaai.v32i1.11671. [25] Lin Lawrence Guo, Maryann Calligan, Emily Vettese, Sadie Cook, George Gagnidze, Oscar Han, Jiro Inoue, Joshua Lemmon, Johnson Li, Medhat Roshdi, Bohdan Sadovy, Steven Wallace, and Lillian Sung. Development and validation of the SickKids enterprise-wide data in Azure repository (SEDAR). Heliyon, 9(11):e21586, 2023. ISSN 2405-8440. doi: 10.1016/j.heliyon.2023.e21586. URL https://www. sciencedirect.com/science/article/pii/S2405844023087947. [26] Bert Arnrich, Edward Choi, Jason Alan Fries, Matthew B. A. McDermott, Jungwoo Oh, Tom Pollard, Nigam Shah, Ethan Steinberg, Michael Wornow, and Robin van de Water. Medical event data standard (MEDS): Facilitating machine learning for health. In ICLR 2024 Workshop on Learning from Time Series For Health, 2024. URL https://openreview.net/forum?id=IsHy2ebjIG. [27] Ethan Steinberg, Michael Wornow, Suhana Bedi, Jason Alan Fries, Matthew B. A. McDermott, and Nigam H. Shah. meds_reader: A fast and efficient EHR processing library. arXiv, 2409.09095, 2024. URL https://arxiv.org/abs/2409.09095. [28] A. E. W. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow, L. H. Lehman, L. A. Celi, and R. G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Sci Data, 10(1):1, 2023. ISSN 2052-4463. doi: 10.1038/s41597-022-01899-x. [29] OHDSI. MIMIC (medical information mart for intensive care) - OMOP CDM ETL repository, 2024. URL https://github.com/OHDSI/MIMIC.
17
Portable EHR Representations
Preprint
[30] Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220, 2000. doi: 10.1161/01.CIR.101.23.e215. URL https://www.ahajournals. org/doi/abs/10.1161/01.CIR.101.23.e215. [31] François Remy, Kris Demuynck, and Thomas Demeester. BioLORD-2023: Semantic textual representations fusing large language models and clinical knowledge graph insights. Journal of the American Medical Informatics Association, 31(9):1844–1855, 2024. doi: 10.1093/jamia/ocae029. URL https://doi.org/10.1093/jamia/ocae029. [32] Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-Embedding: Multilinguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, 2024. doi: 10.18653/v1/2024.findings-acl.137. URL https://aclanthology.org/2024.findings-acl.137/. [33] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv, 2506.05176, 2025. URL https: //arxiv.org/abs/2506.05176. [34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017. URL https://papers.nips.cc/paper_files/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html. [35] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel HerbertVoss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020. URL https://papers.nips.cc/paper_files/paper/2020/hash/ 1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html. [36] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. arXiv, 2104.09864, 2021. doi: 10.48550/arXiv.2104.09864. URL https://arxiv.org/abs/2104.09864. [37] Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv, 1904.10509, 2019. URL https://arxiv.org/abs/1904.10509. [38] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7. [39] Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling data-constrained language models. Advances in Neural Information Processing Systems, 36, 2023. URL https://proceedings.neurips.cc/paper_ files/paper/2023/hash/9d89448b63ce1e2e8dc7af72c984c196-Abstract-Conference.html. [40] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. Scikitlearn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011. URL https://jmlr.org/papers/v12/pedregosa11a.html. [41] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv, 2001.08361, 2020. URL https://arxiv.org/abs/2001.08361. [42] Frank Wilcoxon. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80–83, 1945. doi: 10.2307/3001968. [43] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C. J. Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde,
18
Portable EHR Representations
Preprint
Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, Aditya Vijaykumar, Alessandro Pietro Bardelli, Alex Rothberg, Andreas Hilboll, Andreas Kloeckner, Anthony Scopatz, Antony Lee, Ariel Rokem, C. Nathan Woods, Chad Fulton, Charles Masson, Christian Häggström, Clark Fitzgerald, David A. Nicholson, David R. Hagen, Dmitrii V. Pasechnik, Emanuele Olivetti, Eric Martin, Eric Wieser, Fabrice Silva, Felix Lenders, Florian Wilhelm, G. Young, Gavin A. Price, Gert-Ludwig Ingold, Gregory E. Allen, Gregory R. Lee, Hervé Audren, Irvin Probst, Jörg P. Dietrich, Jacob Silterra, James T. Webber, Janko Slavič, Joel Nothman, Johannes Buchner, Johannes Kulick, Johannes L. Schönberger, José Vinícius de Miranda Cardoso, Joscha Reimer, Joseph Harrington, Juan Luis Cano Rodríguez, Juan Nunez-Iglesias, Justin Kuczynski, Kevin Tritz, Martin Thoma, Matthew Newville, Matthias Kümmerer, Maximilian Bolingbroke, Michael Tartre, Mikhail Pak, Nathaniel J. Smith, Nikolai Nowaczyk, Nikolay Shebanov, Oleksandr Pavlyk, Per A. Brodtkorb, Perry Lee, Robert T. McGibbon, Roman Feldbauer, Sam Lewis, Sam Tygier, Scott Sievert, Sebastiano Vigna, Stefan Peterson, Surhud More, Tadeusz Pudlik, Takuya Oshima, et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 17(3):261–272, 2020. ISSN 1548-7105. doi: 10.1038/s41592-019-0686-2. URL https://doi.org/10.1038/s41592-019-0686-2. [44] Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6:65–70, 1979. [45] Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. MTEB: Massive text embedding benchmark. In Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, 2023. doi: 10.18653/v1/2023.eacl-main.148. URL https://aclanthology.org/2023. eacl-main.148/. [46] Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, 2020. doi: 10.18653/v1/2020.acl-main.740. URL https://aclanthology.org/2020.acl-main.740/. [47] Lingling Xu, Haoran Xie, S. Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(6):6107–6126, 2026. doi: 10.1109/TPAMI.2026.3657354. URL https://doi.org/10.1109/TPAMI.2026.3657354. [48] Zilin Jing, Vincent Jeanselme, Yuta Kobayashi, Simon A. Lee, Chao Pang, Aparajita Kashyap, Yanwei Li, Xinzhuo Jiang, and Shalmali Joshi. One loss to rule them all: Marked time-to-event for structured EHR foundation models. arXiv, 2602.00541, 2026. URL https://arxiv.org/abs/2602.00541.
19
Portable EHR Representations
Preprint
Supplementary Material Supplementary Table S1. Comparison of PORTER with related text-based approaches for structured EHR modeling Approach Text representation granularitya Hegselmann Patient-level et al. [15] LRRL [14] Patient-level DescEmb [19] GenHPF [22] Hur et al. 2026 [23] GRASP [18] GAME [20] Attrach et al. [21] PORTER
Methodological features Text EHR encoderb pretraining objective
Code- or Event-level Code- or Event-level Code- or Event-level
Frozen (pretrained) Frozen (pretrained) Train or fine-tune Train or fine-tune Train or fine-tune
Code- or Event-level Code- or Event-level Code- or Event-level Code- or Event-level
Frozen (pretrained) Frozen (pretrained) Frozen (pretrained) Frozen (pretrained)
Evaluation Frozen patient representation reused across tasks
—
—
Explicit numeric value handling Partial
Cross- Crosssite vocabulary (same patient)c ✓ —
—
—
Partial
—
—
—
—
Partial
✓
—
Masked + contrastive —
—
Partial
✓
—
—
Partial
✓
—
—
—
—
✓
—
—
—
—
✓
—
—
—
✓
—
—
Autoregressive (next-event prediction)
✓
✓
✓
✓
Symbols. ✓indicates present; — indicates absent or not applicable. a
Patient-level serializes the whole record into a single text document. Code- or event-level renders each code or clinical event as text. For GAME, text is one of several inputs to its code embeddings. b
Reflects each method's primary or representative encoder configuration. Frozen (pretrained): a pretrained text encoder used without weight updates; train or fine-tune: encoder weights updated, from random or pretrained initialization. c
The cross-vocabulary evaluation isolates vocabulary shift by holding patients, labels, and clinical events fixed and changing only the naming system Abbreviations: EHR, electronic health record; FM, foundation model; LRRL, large language model rubric representation learning; GRASP, Generalizable Risk Assessment with Semantic Projection; GAME, graph alignment for multi-institutional EHR data; PORTER, Portable EHR Representations.
20
Portable EHR Representations
Preprint
Supplementary Figure S1. Cohort construction for pretraining and downstream evaluation SickKids EHR Dataset as of 2026-04-07 Clinical events: 2018-06-02 to 2026-04-07 (N=2,100,646)
All inpatient admissions (n=124,297)
Data Split
Eligibility
All patients
• •
Patient-level random split (90/10)
Train (n=1,911,000) Events up to 2024-12-31
Validation (n=189,646) Events up to 2025-03-31
Pretraining Cohort Unit = patients
Task-specific exclusion Age < 28 days at prediction time Death, discharge, or outcome on or before prediction time Temporal Split Train: ≤ 2024-12-31 Validation: 2025-01-01 to 2025-03-31 Test: ≥ 2025-04-01 Train (n=52,308 to 84,631)
Validation (n=1,991 to 3,244)
Test (n=8,149 to 13,013)
Clinical Prediction Task Cohort Unit = admissions
Pretraining (left) used a patient-level random split, with different temporal cutoffs for training and validation to evaluate models on more recent clinical data during pretraining. Downstream evaluation (right) used admission-level temporal splits across 74 clinical prediction tasks, aligned to the same calendar cutoffs as pretraining. The unit of analysis was patients for pretraining and admissions for downstream prediction. Abbreviations: EHR, electronic health records; SickKids, The Hospital for Sick Children.
21
Portable EHR Representations
Preprint
Supplementary Table S2. SickKids and MIMIC task cohort statistics Task SickKids Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume Low mean platelet volume High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl Any inotrope
Total admissions
Total patients
Positive cases
Prevalence (%)
97,639 92,090
56,860 54,100
3,326 7,367
3.4 8.0
95,685 99,209 97,168 99,223 98,241 76,757 98,251
55,569 57,382 56,373 57,383 57,018 43,698 57,179
1,782 470 7,758 848 2,546 13,623 8,309
1.9 0.5 8.0 0.9 2.6 17.7 8.5
88,431 96,513 98,880 98,868 97,799 97,350 99,249 99,254
52,133 56,325 57,273 57,258 56,743 56,495 57,382 57,386
10,411 11,981 2,352 1,428 3,398 6,015 181 223
11.8 12.4 2.4 1.4 3.5 6.2 0.2 0.2
90,111 95,165 91,485 97,791 93,615 97,525 92,867 95,409 86,503 95,707 95,576 98,114 98,550 94,099 90,823 97,245 98,512 97,647 94,819 98,626 96,220 96,050 92,243 91,134 95,319 94,304 90,253 86,739 97,176 95,557 97,735 95,514 94,750 95,706 98,306 94,637 99,091 98,593 98,066 98,765 92,870 98,189 98,394 67,901
53,055 55,980 53,617 56,927 54,670 56,694 54,773 55,740 51,946 56,292 55,804 57,037 57,049 55,398 53,949 56,684 57,047 56,802 55,317 57,033 55,753 55,989 54,145 53,508 55,879 55,228 53,174 51,197 56,699 55,956 56,967 55,840 55,709 56,008 56,978 55,260 57,330 57,026 56,786 57,089 54,447 56,872 56,970 40,398
12,829 9,835 11,348 7,393 13,288 4,664 13,948 4,335 19,571 6,733 5,066 4,256 3,257 12,402 13,118 5,804 2,744 4,904 8,600 2,357 1,821 3,734 7,322 10,158 8,529 8,103 14,367 12,280 4,174 5,327 3,579 11,950 9,235 7,727 2,653 5,682 596 2,527 3,542 859 12,666 3,051 2,829 23,556
14.2 10.3 12.4 7.6 14.2 4.8 15.0 4.5 22.6 7.0 5.3 4.3 3.3 13.2 14.4 6.0 2.8 5.0 9.1 2.4 1.9 3.9 7.9 11.1 8.9 8.6 15.9 14.2 4.3 5.6 3.7 12.5 9.7 8.1 2.7 6.0 0.6 2.6 3.6 0.9 13.6 3.1 2.9 34.7
62,448 98,452 94,604 88,672 76,125 79,737 67,830 76,869 79,101 95,062
39,290 57,331 57,184 54,112 45,350 46,577 39,184 44,403 45,636 55,180
19,768 2,424 3,460 7,545 15,972 11,147 18,109 13,733 13,959 2,154
31.7 2.5 3.7 8.5 21.0 14.0 26.7 17.9 17.6 2.3
22
Portable EHR Representations
Task Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality MIMIC Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High lymphocyte Low lymphocyte Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High platelet Low platelet High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
Total admissions
Total patients
Preprint
Positive cases
Prevalence (%)
99,366 100,149 99,258
57,437 57,767 57,387
22,046 17,168 580
22.2 17.1 0.6
44,613 57,091 54,519 58,321 58,311 55,002 35,499 54,447 54,403 56,924 52,218 57,321 57,967 52,680 41,930 57,390 54,759 55,915 56,570 39,084 57,967 48,973 46,286 53,539 54,017 53,125 55,094 56,066 58,307 58,083 57,909 57,037 57,911
34,696 43,323 41,165 43,961 43,924 41,888 28,992 41,392 41,587 43,179 39,892 43,351 43,617 40,039 32,145 43,241 41,526 42,348 42,678 30,204 43,715 37,556 35,609 40,602 40,906 40,161 41,749 42,212 43,895 43,736 43,615 42,883 43,667
12,036 4,090 3,470 588 327 2,684 15,397 4,273 2,392 3,169 8,889 2,205 1,304 8,128 10,430 3,900 8,457 6,168 8,118 14,428 3,732 6,815 9,241 7,748 4,752 5,246 4,974 2,611 475 1,632 2,267 1,508 1,728
27.0 7.2 6.4 1.0 0.6 4.9 43.4 7.8 4.4 5.6 17.0 3.8 2.2 15.4 24.9 6.8 15.4 11.0 14.4 36.9 6.4 13.9 20.0 14.5 8.8 9.9 9.0 4.7 0.8 2.8 3.9 2.6 3.0
58,513 58,512 58,513
44,055 44,055 44,055
17,218 3,143 1,741
29.4 5.4 3.0
Abbreviations: MIMIC, Medical Information Mart for Intensive Care; SickKids, The Hospital for Sick Children.
23
Portable EHR Representations
Preprint
Supplementary Table S3. OMOP-derived event description templates and source columns Category demographics demographics demographics Measurement* Condition Observation Procedure Drug* note specimen visit
Slot mapping to OMOP-derived fieldsb
Template segments a
Patient Birth Date Patient Death Datea Patient Sex: {sex} Measurement: {name} Result: {result} Condition: {name} Observation: {name} Procedure: {name} Drug: {name} Route: {route} Note: {name} Specimen: {name} Visit: {name}
None None sex: gender_concept_name name: measurement_concept_name result: meas_value_concept_name name: condition_concept_name name: observation_concept_name name: procedure_concept_name name: drug_concept_name route: drug_route_concept_name name: note_concept_name name: spec_anatomic_site_concept_name name: visit_concept_name
*Template segments whose source columns are NULL are dropped. For example, measurement events emit "Measurement: {name}. Result: {result}" when meas_value_concept_name is not null, and only "Measurement: {name}" when meas_value_concept_name is null. a
Patient birth date and patient death date were represented as fixed demographic event descriptions.
b
Fields ending in _concept_name are derived fields obtained by joining the corresponding OMOP concept identifier field, such as measurement_concept_id, condition_concept_id, or drug_concept_id, to the OMOP concept table and using the associated concept_name. Analogous SEDAR-derived concept and attribute fields were used to construct event descriptions for cross-vocabulary evaluation. Abbreviations: OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise-wide Data in Azure Repository.
24
Portable EHR Representations
Preprint
Supplementary Figure S2. Pretraining loss curves for PORTER using different text encoders
Cross-entropy loss during pretraining for the four text encoder variants. Training loss was logged every 10 steps. Held-out evaluation loss was logged every 250 steps. Faded traces show raw values. Bold traces show exponential moving averages (span 50 for training, 10 for evaluation). Upper-right insets magnify the last ~7,000 steps so late-training differences are legible.
25
Portable EHR Representations
Preprint
Supplementary Figure S3. Empirical distribution of pre-clip numeric scalar features
Distributions of the log1p fallback scalar (left) and reference-range position scalar (right) are shown before clipping in a sample (n=50,000) from the SickKids training split.
26
Portable EHR Representations
Preprint
Supplementary Table S4. Training, model, and evaluation hyperparameters Hyperparameter
Value
Pretraining optimization and batching Optimizer Learning rate Learning rate scheduler Num epochs Gradient accumulation steps Early stopping *Max clinical events per batch *Min patients per batch Weight decay Max gradient norm (clipping) Warmup steps Adam β1 Adam β2 Floating-point format Transformer backbone Hidden size Num layers Num attention heads Intermediate size Activation **Alternating dense layers **Attention width Linear probe (Logistic Regression) Input preprocessing Solver Regularization Inverse regularization (C) Max iterations
AdamW 5e-4 cosine_with_min_lr 5 2 None 16,384 1 0.05 1.0 150 0.9 0.95 bf16 768 28 12 768 SwiGLU Yes 128 StandardScaler LBFGS L2 1, 0.1, 0.01, 0.001, 0.0001 10,000
* Batches were constructed using a fixed clinical-event budget (16,384 clinical events per batch) with a minimum of one patient per batch to accommodate variable-length patient sequences without padding. This imposes an effective upper bound of 16,384 events per patient sequence (i.e., maximum context window). When multiple patient sequences were in a batch, causal masking prevented attention across patient boundaries, ensuring independent sequence modeling. ** The transformer alternates between global and local self-attention, starting with a global attention layer, followed by a local attention layer (attention width of 128), and repeating this pattern throughout.
27
Portable EHR Representations
Preprint
Supplementary Figure S4. Illustrative serialized patient timeline for the text serialization comparator # Electronic Healthcare Record Current time: [2024-01-01] ## Patient Demographics Age: 1 months Sex: Female ## Recent Body Metrics - Body weight: 3.7 kilogram on 2023-12-22; 3.9 kilogram on 2023-12-21 ## Recent Vital Signs - Respiratory rate: 41.7 on 2023-12-23; 47.3 on 2023-12-23; 39.8 on 2023-12-23 - Body temperature: 98.4 on 2023-12-23; 101.5 on 2023-12-23; 99.1 on 2023-12-23 - Oxygen saturation (SpO2): 93.9 percent on 2023-12-23; 98.4 percent on 2023-12-23; 92.8 percent on 2023-12-23 ## Recent Lab Results - Hemoglobin: 96.3 gram per liter (low) on 2023-12-22 - Hematocrit: 0.3 liter per liter (low) on 2023-12-22 - Leukocytes (WBC): 6.4 billion per liter (low) on 2023-12-22 - Platelets: 206.3 billion per liter (low) on 2023-12-22 - Sodium: 137.1 millimole per liter (normal) on 2023-12-22 - Potassium: 3.7 millimole per liter (normal) on 2023-12-22 - Glucose: 5.6 millimole per liter (high) on 2023-12-22 - Creatinine: 38.6 micromole per liter (high) on 2023-12-22 ## Past Medical Visits - Emergency Room and Inpatient Visit on 2023-12-21 - Health examination on 2023-12-23 ## General Medical Events - Disturbance of temperature regulation of newborn, unspecified ## Detailed Past Medical Visits (most recent first) ### Health examination on 2023-12-23 ### Emergency Room and Inpatient Visit on 2023-12-21 #### Conditions - Fever, unspecified […] #### Medications - acetaminophen: Oral […] #### Procedures - Plain X-ray of chest […]
Text serialization of a single synthetic patient's event stream into Markdown for the text serialization comparator. The full document is prefixed with the task-specific instruction (Supplementary Table S4) before encoding with Qwen3-Embedding-8B. Abbreviation: LLM, large language model.
28
Portable EHR Representations
Preprint
Supplementary Table S5. Task-specific retrieval instructions used by the text serialization comparator Task Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume Low mean platelet volume High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl
Query (verbatim) will the patient need a platelet transfusion will the patient need a red blood cell transfusion will the patient require invasive intubation will the patient need a gastrostomy tube placement will the patient need an echocardiogram will the patient need a pulmonary function test will the patient need a lumbar puncture will the patient need surgery will the patient need an image-guided therapy procedure will the patient need a chest x-ray will the patient need an abdominal ultrasound will the patient need a chest CT scan will the patient need an abdominal CT scan will the patient need a head CT scan will the patient need a head MRI will the patient need a whole body MRI will the patient need a PET scan will the patient have a high white blood cell count will the patient have a low white blood cell count will the patient have a high absolute neutrophil count will the patient have a low absolute neutrophil count will the patient have a high band neutrophil count will the patient have a high lymphocyte count will the patient have a low lymphocyte count will the patient have a high hemoglobin level will the patient have a low hemoglobin level will the patient have a high mean corpuscular volume will the patient have a low mean corpuscular volume will the patient have a high absolute reticulocyte count will the patient have a low absolute reticulocyte count will the patient have a high platelet count will the patient have a low platelet count will the patient have a high immature platelet fraction will the patient have a low immature platelet fraction will the patient have a high mean platelet volume will the patient have a low mean platelet volume will the patient have a high fibrinogen level will the patient have a low fibrinogen level will the patient have a high partial thromboplastin time will the patient have a high international normalized ratio will the patient have a high sodium level will the patient have a low sodium level will the patient have a high potassium level will the patient have a low potassium level will the patient have a high blood glucose level will the patient have a low blood glucose level will the patient have a high creatinine level will the patient have a high urea level will the patient have a low albumin level will the patient have a high alanine aminotransferase level will the patient have a high aspartate aminotransferase level will the patient have a high lactate dehydrogenase level will the patient have a high bilirubin level will the patient have a high cholesterol level will the patient have a high triglyceride level will the patient have a high ferritin level will the patient have a high creatine kinase level will the patient have a high C-reactive protein level will the patient have a high erythrocyte sedimentation rate will the patient have a low partial pressure of oxygen will the patient have low oxygen saturation will the patient need antibacterial medication will the patient need antifungal medication will the patient need chemotherapy will the patient need antiepileptic medication will the patient need glucocorticoid medication will the patient need dexamethasone will the patient need opioid medication will the patient need morphine will the patient need fentanyl
29
Portable EHR Representations
Task Any inotrope Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
Query (verbatim) will the patient need inotrope or vasopressor medication will the patient stay in the hospital for more than 7 days will the patient be readmitted to the hospital within 30 days will the patient die during this hospital admission
Abbreviations: LLM, large language model.
30
Preprint
Portable EHR Representations
Preprint
Supplementary Table S6. Pretraining compute and parameter size by model Model*
Frozen lookup params
Trainable input params
PORTER Fixed-Vocab FM PORTER-Qwen3 PORTER-BGE-M3 PORTER-Random PORTER-NoNum PORTER-NumText
15.12 M 0 80.63 M 20.16 M 80.63 M 15.12 M 876.74 M
1.64 M 75.50 M 4.20 M 1.84 M 4.20 M 1.18 M 1.18 M
Total trainable params 192.87 M 266.72 M 195.42 M 193.06 M 195.42 M 192.41 M 192.41 M
Total FLOPs 1.01 × 1018 9.98 × 1017 1.02 × 1018 1.01 × 1018 1.02 × 1018 1.00 × 1018 1.00 × 1018
* The transformer backbone (115.63 M), next-event prediction head (75.60 M), and number of events seen (894 M) are fixed across all models. All parameter counts are reported in millions (M). Differences in trainable parameters arise from the input pathway: 1) the learnable joint embedding table (i.e., Fixed-Vocab FM, 75.50 M); 2) the projection MLP from each text encoder's native dimension to the backbone hidden dimension; 3) the FiLM numeric pathway present in all PORTER variants except PORTER-NoNum and PORTER-NumText, which render numeric values as text or omit them entirely. Abbreviations: FiLM, feature-wise linear modulation; FLOPs, floating-point operations; FM, foundation model; M, million; MLP, multilayer perceptron.
31
Supplementary Table S7. Per-task AUROC for PORTER and Fixed-Vocab FM across in-domain, cross-vocabulary, and cross-site settings Task
PORTER
Cross-vocabulary (SK) Fixed-Vocab Target-vocabulary FMa Fixed-Vocab FM referenceb
PORTER
Cross-site (MIMIC) Fixed-Vocab Target-site FM Fixed-Vocab FM referencec
0.970 0.926
0.944 0.891
---
0.961 0.919
---
---
---
0.924 0.885 0.894 0.966 0.947 0.891 0.864
0.925 0.886 0.892 0.966 0.941 0.887 0.864
0.894 0.855 0.869 0.941 0.916 0.850 0.815
--------
0.931 0.903 0.890 0.962 0.942 0.894 0.860
--------
--------
--------
0.820 0.841 0.868
0.820 0.847 0.866
0.787 0.810 0.818
----
0.824 0.836 0.852
----
----
----
0.894
0.891
0.824
--
0.883
--
--
--
0.910
0.909
0.869
--
0.911
--
--
--
0.908 0.948 0.897
0.912 0.944 0.901
0.875 0.903 0.823
----
0.912 0.913 0.801
----
----
----
0.834 0.920 0.839
0.830 0.916 0.840
0.781 0.883 0.788
----
0.795 0.895 0.814
0.760 0.869 0.795
0.751 0.876 0.761
0.803 0.905 0.818
0.934
0.934
0.906
--
0.917
0.853
0.852
0.886
0.872 0.852 0.894 0.861 0.887 0.892
0.874 0.848 0.895 0.848 0.882 0.895
0.845 0.816 0.871 0.818 0.855 0.860
-------
0.868 0.830 0.887 0.839 0.875 0.885
-0.713 0.816 -0.859 0.793
-0.680 0.793 -0.850 0.839
-0.737 0.831 -0.876 0.813
0.856
0.838
0.779
--
0.804
0.806
0.856
0.766
0.898 0.893 0.841 0.877 0.892
0.894 0.887 0.839 0.880 0.885
0.855 0.851 0.805 0.846 0.855
------
0.884 0.851 0.827 0.875 0.882
--0.792 0.813 --
--0.773 0.800 --
--0.835 0.852 --
0.870
0.873
0.844
--
0.851
--
--
--
0.902
0.902
0.880
--
0.907
--
--
--
Preprint
0.967 0.929
Portable EHR Representations
32
Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume
In-domain (SK) PORTER Fixed-Vocab FM
Task
PORTER
Cross-vocabulary (SK) Fixed-Vocab Target-vocabulary FMa Fixed-Vocab FM referenceb
PORTER
Cross-site (MIMIC) Fixed-Vocab Target-site FM Fixed-Vocab FM referencec
0.792 0.891 0.920 0.899
0.797 0.888 0.909 0.901
0.762 0.848 0.886 0.865
-----
0.783 0.884 0.899 0.895
-0.753 0.866 0.867
-0.713 0.843 0.844
-0.794 0.895 0.893
0.869
0.874
0.834
--
0.863
0.884
0.869
0.906
0.851 0.854 0.836 0.868 0.832 0.882 0.914 0.926 0.878 0.885 0.894
0.849 0.854 0.834 0.871 0.834 0.884 0.910 0.928 0.877 0.886 0.896
0.813 0.832 0.807 0.838 0.796 0.855 0.883 0.896 0.849 0.853 0.862
------------
0.840 0.841 0.833 0.865 0.830 0.873 0.896 0.909 0.869 0.865 0.878
0.818 0.825 0.834 0.819 0.927 0.783 0.862 0.877 0.849 0.800 0.800
0.812 0.811 0.815 0.823 0.920 0.761 0.822 0.864 0.824 0.810 0.784
0.836 0.834 0.845 0.835 0.935 0.812 0.868 0.887 0.872 0.826 0.829
0.913
0.910
0.875
--
0.897
0.827
0.810
0.854
0.893 0.906 0.876 0.870 0.905 0.849 0.932
0.885 0.908 0.858 0.870 0.911 0.851 0.936
0.862 0.864 0.817 0.834 0.862 0.814 0.884
--------
0.879 0.882 0.843 0.856 0.893 0.840 0.903
0.834 0.785 0.773 0.833 0.809 0.793 --
0.811 0.754 0.770 0.801 0.802 0.763 --
0.862 0.861 0.830 0.852 0.853 0.829 --
0.954 0.815
0.958 0.817
0.943 0.766
---
0.958 0.823
---
---
---
0.864 0.942 0.974 0.870 0.842 0.842 0.853 0.836 0.843 0.926
0.863 0.949 0.977 0.866 0.842 0.847 0.855 0.832 0.847 0.930
0.831 0.925 0.952 0.832 0.806 0.802 0.814 0.799 0.807 0.880
-----------
0.854 0.941 0.969 0.866 0.844 0.848 0.856 0.837 0.846 0.930
-----------
-----------
-----------
0.834
0.835
0.795
--
0.835
0.811
0.789
0.835
0.793
0.794
0.775
--
0.786
0.833
0.819
0.844
0.941
0.942
0.905
--
0.937
0.895
0.886
0.932
Fixed-Vocab FM was not evaluated in cross-vocabulary transfer because SEDAR composite event tokens had no learned input embeddings.
b
The target-vocabulary reference is a Fixed-Vocab FM trained directly on the SEDAR vocabulary.
Preprint
a
Portable EHR Representations
33
Low mean platelet volume High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl Any inotrope Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
In-domain (SK) PORTER Fixed-Vocab FM
c
The target-site reference is a Fixed-Vocab FM trained directly on MIMIC. MIMIC cross-site columns include 36 labels.
Abbreviations: AUROC, area under the receiver operating characteristic curve; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids.
Portable EHR Representations
34
Preprint
Portable EHR Representations
Preprint
Supplementary Figure S5. Sample efficiency curves across in-domain, cross-vocabulary, and cross-site settings for PORTER and Fixed-Vocab FM
Curves summarize linear-probe performance as labeled training examples per task increase. PORTER is compared with Fixed-Vocab FM when it can be evaluated. The cross-vocabulary panel shows PORTER alone because Fixed-Vocab FM was not evaluated in this setting. SEDAR composite event tokens had no learned input embeddings. Abbreviations: FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids.
35
Portable EHR Representations
Preprint
Supplementary Table S8. Full-shot per-task AUROC for PORTER and text serialization comparator Task Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume Low mean platelet volume High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl
PORTER
Text serialization comparator
0.967 0.929
0.963 0.915
0.924 0.885 0.894 0.966 0.947 0.891 0.864
0.896 0.912 0.882 0.948 0.930 0.872 0.841
0.820 0.841 0.868 0.894 0.910 0.908 0.948 0.897
0.797 0.821 0.845 0.867 0.893 0.891 0.911 0.892
0.834 0.920 0.839 0.934 0.872 0.852 0.894 0.861 0.887 0.892 0.856 0.898 0.893 0.841 0.877 0.892 0.870 0.902 0.792 0.891 0.920 0.899 0.869 0.851 0.854 0.836 0.868 0.832 0.882 0.914 0.926 0.878 0.885 0.894 0.913 0.893 0.906 0.876 0.870 0.905 0.849 0.932 0.954 0.815
0.804 0.903 0.810 0.917 0.854 0.813 0.874 0.848 0.866 0.895 0.836 0.878 0.865 0.833 0.877 0.886 0.858 0.906 0.781 0.858 0.900 0.882 0.838 0.814 0.834 0.819 0.849 0.804 0.849 0.905 0.916 0.852 0.861 0.873 0.882 0.877 0.874 0.817 0.846 0.821 0.823 0.916 0.949 0.779
0.864 0.942 0.974 0.870 0.842 0.842 0.853 0.836 0.843
0.841 0.935 0.968 0.844 0.806 0.809 0.821 0.799 0.809
36
Portable EHR Representations
Task Any inotrope Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
PORTER
Preprint
Text serialization comparator
0.926
0.909
0.834 0.793 0.941
0.812 0.796 0.925
Abbreviations: AUROC, area under the receiver operating characteristic curve; LLM, large language model..
37
Supplementary Table S9. Per-task AUROC for PORTER and its text encoder ablations across in-domain, cross-vocabulary, and cross-site settings Task BioLORD
Random
Cross-vocabulary (SK) BioLORD Qwen3 BGEM3
Random
Cross-site (MIMIC)* BioLORD Qwen3 BGEM3
Random
0.968 0.921
0.969 0.924
0.964 0.920
0.944 0.891
0.939 0.893
0.930 0.869
0.902 0.842
---
---
---
---
0.924 0.885 0.894 0.966 0.947 0.891 0.864
0.925 0.884 0.889 0.962 0.946 0.881 0.859
0.908 0.884 0.882 0.967 0.936 0.879 0.855
0.920 0.867 0.889 0.958 0.938 0.881 0.851
0.894 0.855 0.869 0.941 0.916 0.850 0.815
0.884 0.842 0.864 0.935 0.898 0.852 0.818
0.848 0.784 0.844 0.897 0.866 0.816 0.778
0.813 0.765 0.800 0.838 0.816 0.766 0.734
--------
--------
--------
--------
0.820 0.841 0.868
0.820 0.836 0.860
0.818 0.836 0.855
0.822 0.839 0.849
0.787 0.810 0.818
0.785 0.802 0.808
0.765 0.774 0.801
0.713 0.736 0.753
----
----
----
----
0.894
0.891
0.891
0.883
0.824
0.847
0.793
0.738
--
--
--
--
0.910
0.912
0.907
0.903
0.869
0.863
0.793
0.722
--
--
--
--
0.908 0.948 0.897
0.905 0.940 0.917
0.906 0.938 0.916
0.905 0.952 0.880
0.875 0.903 0.823
0.855 0.845 0.829
0.812 0.874 0.795
0.739 0.778 0.750
----
----
----
----
0.834 0.920 0.839
0.833 0.919 0.838
0.831 0.918 0.836
0.835 0.919 0.838
0.781 0.883 0.788
0.775 0.875 0.785
0.749 0.867 0.754
0.697 0.847 0.721
0.760 0.869 0.795
0.765 0.891 0.785
0.756 0.876 0.776
0.758 0.875 0.758
0.934
0.934
0.933
0.931
0.906
0.902
0.884
0.864
0.853
0.860
0.850
0.856
0.872 0.852 0.894 0.861 0.887 0.892
0.870 0.848 0.895 0.847 0.886 0.904
0.871 0.849 0.894 0.844 0.882 0.898
0.869 0.849 0.892 0.840 0.881 0.887
0.845 0.816 0.871 0.818 0.855 0.860
0.838 0.817 0.864 0.809 0.856 0.854
0.822 0.798 0.849 0.799 0.832 0.852
0.809 0.761 0.829 0.777 0.805 0.825
-0.713 0.816 -0.859 0.793
-0.716 0.803 -0.862 0.808
-0.689 0.806 -0.865 0.808
-0.704 0.780 -0.855 0.812
0.856
0.866
0.863
0.848
0.779
0.781
0.765
0.724
0.806
0.823
0.823
0.821
0.898 0.893 0.841 0.877 0.892
0.890 0.876 0.834 0.876 0.884
0.891 0.886 0.836 0.879 0.889
0.894 0.872 0.841 0.888 0.891
0.855 0.851 0.805 0.846 0.855
0.855 0.839 0.799 0.838 0.853
0.846 0.821 0.784 0.825 0.844
0.824 0.793 0.753 0.799 0.820
--0.792 0.813 --
--0.792 0.821 --
--0.788 0.808 --
--0.773 0.808 --
0.870
0.861
0.868
0.859
0.844
0.827
0.825
0.794
--
--
--
--
0.902
0.900
0.903
0.901
0.880
0.876
0.865
0.835
--
--
--
--
0.792
0.778
0.796
0.802
0.762
0.754
0.735
0.694
--
--
--
--
Preprint
0.967 0.929
Portable EHR Representations
38
Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume Low mean platelet volume
In-domain (SK) Qwen3 BGEM3
Task BioLORD
Random
Cross-vocabulary (SK) BioLORD Qwen3 BGEM3
Random
Cross-site (MIMIC)* BioLORD Qwen3 BGEM3
Random
0.891 0.920 0.899
0.894 0.913 0.899
0.892 0.904 0.901
0.882 0.907 0.902
0.848 0.886 0.865
0.832 0.878 0.869
0.823 0.840 0.836
0.774 0.823 0.808
0.753 0.866 0.867
0.770 0.869 0.863
0.760 0.875 0.861
0.742 0.867 0.847
0.869
0.871
0.868
0.870
0.834
0.826
0.799
0.769
0.884
0.882
0.883
0.873
0.851 0.854 0.836 0.868 0.832 0.882 0.914 0.926 0.878 0.885 0.894
0.845 0.855 0.829 0.868 0.834 0.881 0.916 0.924 0.875 0.885 0.899
0.841 0.851 0.833 0.863 0.833 0.881 0.912 0.930 0.874 0.886 0.894
0.842 0.849 0.829 0.864 0.832 0.876 0.913 0.924 0.873 0.882 0.894
0.813 0.832 0.807 0.838 0.796 0.855 0.883 0.896 0.849 0.853 0.862
0.806 0.809 0.797 0.838 0.796 0.836 0.863 0.889 0.846 0.831 0.836
0.786 0.797 0.785 0.818 0.774 0.830 0.834 0.867 0.823 0.808 0.816
0.750 0.778 0.764 0.778 0.750 0.801 0.799 0.829 0.804 0.789 0.793
0.818 0.825 0.834 0.819 0.927 0.783 0.862 0.877 0.849 0.800 0.800
0.819 0.815 0.828 0.811 0.925 0.788 0.850 0.872 0.849 0.810 0.807
0.819 0.821 0.828 0.808 0.924 0.786 0.847 0.873 0.843 0.803 0.797
0.810 0.799 0.816 0.805 0.917 0.773 0.817 0.851 0.842 0.794 0.784
0.913
0.910
0.906
0.905
0.875
0.855
0.843
0.841
0.827
0.829
0.830
0.821
0.893 0.906 0.876 0.870 0.905 0.849 0.932
0.890 0.890 0.861 0.865 0.901 0.853 0.934
0.889 0.899 0.845 0.864 0.904 0.851 0.933
0.892 0.907 0.862 0.871 0.912 0.855 0.936
0.862 0.864 0.817 0.834 0.862 0.814 0.884
0.855 0.823 0.798 0.833 0.853 0.807 0.885
0.834 0.833 0.790 0.811 0.817 0.785 0.856
0.798 0.702 0.728 0.785 0.755 0.742 0.823
0.834 0.785 0.773 0.833 0.809 0.793 --
0.853 0.782 0.774 0.830 0.814 0.786 --
0.836 0.790 0.768 0.831 0.806 0.799 --
0.824 0.789 0.777 0.819 0.804 0.758 --
0.954 0.815
0.956 0.810
0.954 0.804
0.952 0.805
0.943 0.766
0.946 0.764
0.920 0.739
0.901 0.684
---
---
---
---
0.864 0.942 0.974 0.870 0.842 0.842 0.853 0.836 0.843 0.926
0.863 0.945 0.968 0.862 0.841 0.839 0.845 0.830 0.837 0.933
0.862 0.945 0.972 0.862 0.839 0.844 0.845 0.826 0.837 0.937
0.858 0.934 0.966 0.860 0.836 0.837 0.840 0.823 0.839 0.922
0.831 0.925 0.952 0.832 0.806 0.802 0.814 0.799 0.807 0.880
0.817 0.919 0.948 0.821 0.788 0.797 0.807 0.786 0.796 0.896
0.782 0.904 0.938 0.791 0.753 0.756 0.784 0.757 0.778 0.839
0.726 0.884 0.915 0.733 0.719 0.725 0.727 0.708 0.733 0.810
-----------
-----------
-----------
-----------
0.834
0.828
0.827
0.830
0.795
0.778
0.762
0.724
0.811
0.814
0.807
0.796
0.793
0.793
0.794
0.789
0.775
0.777
0.768
0.744
0.833
0.826
0.832
0.814
0.941
0.935
0.940
0.931
0.905
0.878
0.903
0.828
0.895
0.892
0.900
0.884
Abbreviations: AUROC, area under the receiver operating characteristic curve; MIMIC, Medical Information Mart for Intensive Care; SickKids, The Hospital for Sick Children; SK, SickKids.
Preprint
*MIMIC cross-site columns include 36 labels.
Portable EHR Representations
39
High fibrinogen Low fibrinogen High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl Any inotrope Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
In-domain (SK) Qwen3 BGEM3
Portable EHR Representations
Preprint
Supplementary Figure S6. Sample efficiency curves across in-domain, cross-vocabulary, and cross-site settings for PORTER and its text encoder ablations
Curves summarize linear probe performance as labeled training examples per task increase. PORTER is compared against three ablations that hold the FiLM numeric pathway fixed while varying the text encoder: PORTER-Qwen3 (Qwen3-Embedding-8B), PORTER-BGE-M3 (BGE-M3), and PORTER-Random (randomly initialized encoder). Abbreviations: FiLM, feature-wise linear modulation; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids.
40
Portable EHR Representations
Preprint
Supplementary Figure S7. Patient representation geometry under cross-vocabulary transfer for PORTER and its text encoder ablations
For each encoder, patient representations were computed for the same 9,585 unique test patients using OMOP-derived event descriptions and SEDAR-derived event descriptions. Each point plots the cosine distance between a pair of patients under the OMOP-derived representation against the corresponding distance under the SEDAR-derived representation. The scatter plots show a random subsample (200,000 pairs) for visualization. Spearman ρ and the OLS fit summarize how well the encoder preserves patient-level geometry. Higher ρ, slope closer to 1, and intercept closer to 0 indicate stronger preservation of patient-level geometry under vocabulary transfer. Abbreviations: OLS, ordinary least squares; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise-wide Data in Azure Repository.
41
Supplementary Table S10. Per-task AUROC for PORTER and its numeric pathway ablations across in-domain, cross-vocabulary, and cross-site settings Task PORTER
Cross-vocabulary (SK) PORTER PORTERPORTERNoNum NumText
Cross-site (MIMIC)* PORTER PORTERPORTERNoNum NumText
0.963 0.912
0.968 0.919
0.944 0.891
0.946 0.880
0.942 0.881
---
---
---
0.924 0.885 0.894 0.966 0.947 0.891 0.864
0.935 0.872 0.887 0.965 0.934 0.888 0.858
0.919 0.862 0.893 0.965 0.950 0.885 0.858
0.894 0.855 0.869 0.941 0.916 0.850 0.815
0.885 0.795 0.864 0.929 0.914 0.859 0.812
0.885 0.854 0.856 0.928 0.909 0.844 0.815
--------
--------
--------
0.820 0.841 0.868
0.805 0.836 0.860
0.818 0.843 0.864
0.787 0.810 0.818
0.759 0.799 0.815
0.767 0.802 0.818
----
----
----
0.894
0.878
0.885
0.824
0.839
0.832
--
--
--
0.910
0.909
0.914
0.869
0.885
0.870
--
--
--
0.908 0.948 0.897
0.908 0.942 0.894
0.906 0.933 0.909
0.875 0.903 0.823
0.877 0.846 0.912
0.867 0.860 0.803
----
----
----
0.834 0.920 0.839
0.794 0.888 0.809
0.824 0.916 0.830
0.781 0.883 0.788
0.740 0.864 0.761
0.773 0.878 0.788
0.760 0.869 0.795
0.747 0.844 0.788
0.745 0.844 0.782
0.934
0.915
0.935
0.906
0.898
0.908
0.853
0.844
0.822
0.872 0.852 0.894 0.861 0.887 0.892
0.864 0.822 0.884 0.837 0.867 0.869
0.868 0.846 0.892 0.850 0.880 0.891
0.845 0.816 0.871 0.818 0.855 0.860
0.830 0.793 0.859 0.802 0.830 0.835
0.834 0.803 0.864 0.791 0.841 0.845
-0.713 0.816 -0.859 0.793
-0.694 0.805 -0.847 0.755
-0.682 0.791 -0.851 0.752
0.856
0.804
0.856
0.779
0.759
0.765
0.806
0.680
0.663
0.898 0.893 0.841 0.877 0.892
0.889 0.864 0.818 0.868 0.884
0.889 0.877 0.841 0.886 0.893
0.855 0.851 0.805 0.846 0.855
0.843 0.826 0.779 0.833 0.852
0.848 0.834 0.792 0.840 0.854
--0.792 0.813 --
--0.770 0.798 --
--0.775 0.800 --
0.870
0.862
0.872
0.844
0.841
0.837
--
--
--
0.902 0.792 0.891 0.920
0.894 0.768 0.874 0.912
0.902 0.796 0.885 0.908
0.880 0.762 0.848 0.886
0.870 0.733 0.845 0.879
0.867 0.755 0.836 0.860
--0.753 0.866
--0.744 0.844
--0.719 0.852
Preprint
0.967 0.929
Portable EHR Representations
42
Transfusions Platelet transfusion Red cell transfusion Procedure Invasive intubation Gastrostomy tube Echocardiogram Pulmonary function test Lumbar puncture Surgery Interventional radiology Imaging Plain radiography chest Ultrasound abdomen Computerized tomography chest Computerized tomography abdomen Computerized tomography head MRI head MRI whole body PET Laboratory abnormality High white blood count Low white blood count High absolute neutrophil count Low absolute neutrophil count High bands High lymphocyte Low lymphocyte High hemoglobin Low hemoglobin High mean corpuscular volume Low mean corpuscular volume High reticulocyte count Low reticulocyte count High platelet Low platelet High immature platelet fraction Low immature platelet fraction High mean platelet volume Low mean platelet volume High fibrinogen Low fibrinogen
In-domain (SK) PORTERPORTERNoNum NumText
Task PORTER
Cross-vocabulary (SK) PORTER PORTERPORTERNoNum NumText
Cross-site (MIMIC)* PORTER PORTERPORTERNoNum NumText
0.899
0.892
0.899
0.865
0.853
0.862
0.867
0.852
0.854
0.869
0.860
0.863
0.834
0.820
0.826
0.884
0.876
0.877
0.851 0.854 0.836 0.868 0.832 0.882 0.914 0.926 0.878 0.885 0.894
0.836 0.838 0.826 0.862 0.824 0.878 0.897 0.916 0.870 0.870 0.882
0.846 0.851 0.831 0.869 0.831 0.877 0.901 0.926 0.874 0.883 0.895
0.813 0.832 0.807 0.838 0.796 0.855 0.883 0.896 0.849 0.853 0.862
0.799 0.805 0.792 0.831 0.788 0.839 0.861 0.892 0.843 0.835 0.843
0.805 0.824 0.799 0.832 0.789 0.851 0.875 0.889 0.841 0.839 0.854
0.818 0.825 0.834 0.819 0.927 0.783 0.862 0.877 0.849 0.800 0.800
0.795 0.798 0.817 0.807 0.924 0.777 0.839 0.860 0.839 0.783 0.781
0.798 0.800 0.821 0.800 0.925 0.760 0.833 0.860 0.836 0.780 0.773
0.913
0.902
0.898
0.875
0.868
0.860
0.827
0.811
0.806
0.893 0.906 0.876 0.870 0.905 0.849 0.932
0.887 0.883 0.865 0.852 0.899 0.838 0.914
0.884 0.898 0.858 0.856 0.903 0.844 0.929
0.862 0.864 0.817 0.834 0.862 0.814 0.884
0.850 0.809 0.801 0.831 0.838 0.793 0.880
0.855 0.855 0.772 0.825 0.845 0.802 0.890
0.834 0.785 0.773 0.833 0.809 0.793 --
0.800 0.756 0.758 0.808 0.783 0.790 --
0.814 0.804 0.753 0.802 0.797 0.781 --
0.954 0.815
0.960 0.804
0.956 0.813
0.943 0.766
0.946 0.763
0.948 0.765
---
---
---
0.864 0.942 0.974 0.870 0.842 0.842 0.853 0.836 0.843 0.926
0.858 0.945 0.974 0.863 0.836 0.836 0.845 0.834 0.837 0.931
0.861 0.943 0.973 0.864 0.840 0.847 0.849 0.836 0.841 0.924
0.831 0.925 0.952 0.832 0.806 0.802 0.814 0.799 0.807 0.880
0.823 0.923 0.951 0.816 0.800 0.802 0.814 0.791 0.808 0.896
0.816 0.917 0.953 0.813 0.797 0.794 0.808 0.785 0.803 0.890
-----------
-----------
-----------
0.834
0.823
0.831
0.795
0.777
0.782
0.811
0.805
0.798
0.793
0.791
0.790
0.775
0.778
0.775
0.833
0.836
0.825
0.941
0.929
0.932
0.905
0.889
0.873
0.895
0.877
0.868
Portable EHR Representations
43
High partial thromboplastin time High international normalized ratio High sodium Low sodium High potassium Low potassium High glucose Low glucose High creatinine High urea Low albumin High alanine transaminase High aspartate aminotransferase High lactate dehydrogenase High bilirubin High cholesterol High triglyceride High ferritin High creatinine kinase High C-reactive protein High erythrocyte sedimentation rate Low PaO2 Low SpO2 Medications Any antibacterial Any antifungal Any chemotherapy Any antiepileptics Any glucocorticoid Dexamethasone Any opioid Morphine Fentanyl Any inotrope Clinical outcomes Long length of stay (≥ 7 days) Readmission within 30 days Mortality
In-domain (SK) PORTERPORTERNoNum NumText
*MIMIC cross-site columns include 36 labels. Preprint
Abbreviations: AUROC, area under the receiver operating characteristic curve; MIMIC, Medical Information Mart for Intensive Care; NoNum, no numeric pathway; NumText, numeric-as-text; SickKids, The Hospital for Sick Children; SK, SickKids.
Portable EHR Representations
Preprint
Supplementary Figure S8. Sample efficiency curves across in-domain, cross-vocabulary, and cross-site settings for PORTER and its numeric pathway ablations
Curves summarize linear probe performance as labeled training examples per task increase. PORTER is compared against two numeric pathway ablations while holding the text encoder fixed: PORTER-NumText (numeric value and reference range added to the event text description instead of using FiLM) and PORTERNoNum (no numeric representation). Abbreviations: FiLM, feature-wise linear modulation; FM, foundation model; MIMIC, Medical Information Mart for Intensive Care; SEDAR, SickKids Enterprise-wide Data in Azure Repository; SickKids, The Hospital for Sick Children; SK, SickKids.
44
Portable EHR Representations
Preprint
Supplementary Figure S9. Synonym invariance and numeric sensitivity of event representations passed to the transformer for PORTER and its numeric pathway ablations
Top row: cosine similarity between SEDAR-derived descriptions and OMOP-derived event descriptions, shown as half-violins per domain (within-concept on the left, across-concept distractors on the right). Bottom row: Spearman ρ for each measurement concept between decile index and cosine distance from the first decile, shown as box plots with jittered per-concept points. Lines connect the same measurement concept across the PORTER-NumText and PORTER columns. Line color indicates whether FiLM improves or worsens numeric sensitivity relative to NumText, as shown in the legend. PORTER-NoNum does not consider numeric values, so embeddings across deciles are identical for each concept and ρ is undefined. Abbreviations: FiLM, feature-wise linear modulation; NoNum, no numeric pathway; NumText, numeric values rendered as text; OMOP, Observational Medical Outcomes Partnership; SEDAR, SickKids Enterprise-wide Data in Azure Repository.
45