You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
These checkpoints are derivative works of credentialed intensive-care datasets (MIMIC-IV, eICU-CRD, HiRID via PhysioNet; SICdb). Access is granted only to users who already hold the relevant credentialed-access approvals and agree to the same data-use terms. Each request is reviewed individually by the authors.
Log in or Sign Up to review the conditions and access this model content.
Federated EHR Foundation Model (autoregressive, multi-institutional)
Research checkpoints accompanying Multi-Hospital Electronic Health Record Foundation Models Without Pooling Patient Data: A Comparison of Federated Learning and Inference-Time Ensembling (O. Elemento, Weill Cornell Medicine). Research artifact — not a medical device; not cleared or approved for clinical use.
Model details
- Type: GPT-style decoder-only transformer (~31.6M parameters) trained with a next-token autoregressive objective (strict causal attention, cross-entropy over the full vocabulary) on tokenized patient sequences (diagnosis, procedure, medication, laboratory tokens). RoPE indexed by inter-visit day delta.
- Architecture: 8 layers, 8 heads, d_model 512, d_ff 2048. Regularization (reg2): dropout 0.4, weight decay 0.5, label smoothing 0.1 (train-time only); the four-cohort cross-database FedAvg checkpoint uses a lighter recipe (dropout 0.3, wd 0.3, no label smoothing), which would not converge otherwise.
- Released arms (MIMIC-IV vocabulary):
centralized,fedavg,fedper, andfedper_lora, plus the tokenizer / vocabulary. The five per-site ensemble members are not included in this repository: they are the only arms that show a measurable membership signal under white-box attack, so they are withheld and provided to credentialed researchers on direct request (individual review). The cross-database (Table 1 / Figure 4) results are reproducible from the released code and the aggregate per-run metrics in the code repository. - Developers: Englander Institute for Precision Medicine & Department of Systems and Computational Biomedicine, Weill Cornell Medicine.
Intended use
Reproduction and verification of the manuscript's results, and downstream research on multi-institutional EHR modeling under appropriate governance. Not for clinical decision-making, patient-level deployment, or re-identification.
Training data
MIMIC-IV (Beth Israel Deaconess, US), eICU-CRD (US multi-hospital), HiRID (Switzerland) — all PhysioNet-credentialed — and SICdb (Austria). Diagnosis/ procedure tokens use ICD-10/ICD-9 chapter prefixes; labs mapped to LOINC; medications to ATC where available. Held-out evaluation is leakage-clean (target admission cut at the second-to-last separator; single-stay mortality scored from the first 12-hour block).
Performance (reg2, held-out test)
On MIMIC-IV, per-condition AUROC 0.84–0.91; FedAvg and inference-time ensembling are statistically indistinguishable from centralized training. Across four real ICU databases, the inference-time ensemble tracks centralized pooling while FedAvg degrades by 0.02–0.16 AUROC under cross-database heterogeneity (see manuscript Table 1, Figures 2–4).
Limitations & risks
~31.6M parameters (small vs frontier EHR models); simulated non-IID partitions for the homogeneous-federation experiments; gated release reflects the restricted, credentialed nature of the source data. A membership-inference audit of these checkpoints is reported in the manuscript: the released centralized/FedAvg models show near-chance membership signal (AUROC 0.52-0.55). The individual per-site members can leak under white-box attack (AUROC up to 0.78, an upper bound confounded by per-site distribution shift), so they are withheld from this repository and shared only on direct request to credentialed researchers. The deployed ensemble exposes only its averaged output, whose membership signal is near-random (AUROC 0.55).
Citation
See the accompanying manuscript. Code: https://github.com/oelemento/gpt-ehr-poc