ELF slide encoder

ELF (Ensemble Learning of Foundation models) is a slide-level pathology encoder from the Li Lab at Stanford. It aggregates tile embeddings from five public foundation models into a unified whole-slide representation.

Paper: Ensemble learning of pathology foundation models for precision oncology
Code: github.com/lilab-stanford/ELF
Weights: huggingface.co/luoxd96/ELF

This Hub repo contains the inference-only slide encoder: interpolate → LayerNorm → 8-head gated ABMIL. It is the trained momentum_enc forward used in the paper, not the MoCo training snapshot.

Model details

Model type	Slide-level ABMIL aggregator
Inputs	Patch features (X \in \mathbb{R}^{N \times C}), (C \in {768, 1024, 1280, 1536})
Outputs	features_dim ([1, C]), features ([1, 768]), attention_weights ([1, 1, N])
Pretraining	53,699 WSIs, 20 anatomical sites
Tile encoders	UNI, CONCH v1.5, Prov-GigaPath, Virchow2, H-optimus-0
Magnification	10× patches (CLAM)
License	GPLv3, non-commercial academic use

The paper ensemble is the concatenation of the five features_dim vectors (one per tile foundation model), not an average.

Tile model	--fm	native C	features_dim
UNI	uni	1024	1024
CONCH v1.5	conch_v1_5	768	768
Prov-GigaPath	gigapath	1536	1536
H-optimus-0	h0	1536	1536
Virchow2	virchow2	2560 → 1280 (CLS+mean)	1280

How to use

pip install torch huggingface_hub h5py numpy

Download this file from the Hub (modeling_elf.py) or clone the GitHub repo and use inference/model.py.

import torch
from huggingface_hub import hf_hub_download

# modeling_elf.py from this repo
from modeling_elf import ELFSlideEncoder, preprocess_patch_features

model = ELFSlideEncoder.from_pretrained("luoxd96/ELF", device="cuda")

# patches: [N, C] tile features for one slide and one foundation model
patches = torch.from_numpy(features).float()
patches = preprocess_patch_features(patches, foundation_model="uni")  # virchow2 averages CLS+mean

with torch.inference_mode():
    x = patches.unsqueeze(0).cuda()
    features_dim, features_768, attn = model(x)

# features_dim: [1, C]  — use this for the paper ensemble
# features_768: [1, 768]
# attn:         [1, 1, N]

Batch extract from CLAM-style h5 files (dataset "features"):

git clone https://github.com/lilab-stanford/ELF.git
cd ELF

python inference/extract_multiple_model_slide_embedding.py \
  --checkpoint $(python -c "from huggingface_hub import hf_hub_download; print(hf_hub_download('luoxd96/ELF','elf_slide_encoder.pth'))") \
  --input-dir /path/to/uni/<dataset>/h5_files \
  --output-path /path/to/uni_elf \
  --feature-models uni \
  --gpu 0

Repeat for conch_v1_5, gigapath, virchow2, h0. Concatenate the five features_dim vectors for downstream linear probing, as in evaluation/.

Intended use

Research feature extraction for computational pathology (classification, biomarker prediction, therapy-response studies). This is not a diagnostic device and should not be used for clinical decision-making without independent validation.

Limitations

• Requires pre-extracted tile embeddings from the five foundation models above; it does not encode RGB tiles.
• Virchow2 inputs of dimension ≥ 2560 are averaged as CLS + mean → 1280, matching the paper.

Citation

@misc{luo2026ensemblelearningpathologyfoundation,
  title={Ensemble learning of pathology foundation models for precision oncology},
  author={Xiangde Luo and Xiyue Wang and Feyisope Eweje and Xiaoming Zhang and Juan Luis Gomez Marti and Sarah Cascarino and Sen Yang and Yuchen Li and Ryan Quinton and Jinxi Xiang and Yuanfeng Ji and Zhe Li and Yijiang Chen and Colin Bergstrom and Ted Kim and Francesca Maria Olguin and Kelley Yuan and Matthew Abikenari and Andrew Heider and Sierra Willens and Sanjeeth Rajaram and Robert West and Joel Neal and Adam Schoenfeld and Maximilian Diehn and Chad Vanderbilt and Ruijiang Li},
  year={2026},
  eprint={2508.16085},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2508.16085},
}