File size: 4,620 Bytes
6ec5f7a e22a784 6ec5f7a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | ---
license: gpl-3.0
library_name: pytorch
pipeline_tag: feature-extraction
tags:
- pathology
- histopathology
- computational-pathology
- slide-encoder
- abmil
- foundation-model
- ensemble
---
# ELF slide encoder
**ELF** (Ensemble Learning of Foundation models) is a slide-level pathology encoder from the Li Lab at Stanford. It aggregates tile embeddings from five public foundation models into a unified whole-slide representation.
Paper: [Ensemble learning of pathology foundation models for precision oncology](https://arxiv.org/abs/2508.16085)
Code: [github.com/lilab-stanford/ELF](https://github.com/lilab-stanford/ELF)
Weights: [huggingface.co/luoxd96/ELF](https://huggingface.co/luoxd96/ELF)
This Hub repo contains the **inference-only** slide encoder: interpolate → LayerNorm → 8-head gated ABMIL. It is the trained `momentum_enc` forward used in the paper, not the MoCo training snapshot.
## Model details
| | |
|---|---|
| Model type | Slide-level ABMIL aggregator |
| Inputs | Patch features \(X \in \mathbb{R}^{N \times C}\), \(C \in \{768, 1024, 1280, 1536\}\) |
| Outputs | `features_dim` \([1, C]\), `features` \([1, 768]\), `attention_weights` \([1, 1, N]\) |
| Pretraining | 53,699 WSIs, 20 anatomical sites |
| Tile encoders | UNI, CONCH v1.5, Prov-GigaPath, Virchow2, H-optimus-0 |
| Magnification | 10× patches (CLAM) |
| License | GPLv3, non-commercial academic use |
The paper ensemble is the **concatenation** of the five `features_dim` vectors (one per tile foundation model), not an average.
| Tile model | `--fm` | native `C` | `features_dim` |
|---|---|---|---|
| UNI | `uni` | 1024 | 1024 |
| CONCH v1.5 | `conch_v1_5` | 768 | 768 |
| Prov-GigaPath | `gigapath` | 1536 | 1536 |
| H-optimus-0 | `h0` | 1536 | 1536 |
| Virchow2 | `virchow2` | 2560 → 1280 (CLS+mean) | 1280 |
## How to use
```bash
pip install torch huggingface_hub h5py numpy
```
Download this file from the Hub (`modeling_elf.py`) or clone the [GitHub repo](https://github.com/lilab-stanford/ELF) and use `inference/model.py`.
```python
import torch
from huggingface_hub import hf_hub_download
# modeling_elf.py from this repo
from modeling_elf import ELFSlideEncoder, preprocess_patch_features
model = ELFSlideEncoder.from_pretrained("luoxd96/ELF", device="cuda")
# patches: [N, C] tile features for one slide and one foundation model
patches = torch.from_numpy(features).float()
patches = preprocess_patch_features(patches, foundation_model="uni") # virchow2 averages CLS+mean
with torch.inference_mode():
x = patches.unsqueeze(0).cuda()
features_dim, features_768, attn = model(x)
# features_dim: [1, C] — use this for the paper ensemble
# features_768: [1, 768]
# attn: [1, 1, N]
```
Batch extract from CLAM-style `h5` files (`dataset "features"`):
```bash
git clone https://github.com/lilab-stanford/ELF.git
cd ELF
python inference/extract_multiple_model_slide_embedding.py \
--checkpoint $(python -c "from huggingface_hub import hf_hub_download; print(hf_hub_download('luoxd96/ELF','elf_slide_encoder.pth'))") \
--input-dir /path/to/uni/<dataset>/h5_files \
--output-path /path/to/uni_elf \
--feature-models uni \
--gpu 0
```
Repeat for `conch_v1_5`, `gigapath`, `virchow2`, `h0`. Concatenate the five `features_dim` vectors for downstream linear probing, as in `evaluation/`.
## Intended use
Research feature extraction for computational pathology (classification, biomarker prediction, therapy-response studies). This is **not** a diagnostic device and should not be used for clinical decision-making without independent validation.
## Limitations
- Requires pre-extracted tile embeddings from the five foundation models above; it does not encode RGB tiles.
- Virchow2 inputs of dimension ≥ 2560 are averaged as CLS + mean → 1280, matching the paper.
## Citation
```bibtex
@misc{luo2026ensemblelearningpathologyfoundation,
title={Ensemble learning of pathology foundation models for precision oncology},
author={Xiangde Luo and Xiyue Wang and Feyisope Eweje and Xiaoming Zhang and Juan Luis Gomez Marti and Sarah Cascarino and Sen Yang and Yuchen Li and Ryan Quinton and Jinxi Xiang and Yuanfeng Ji and Zhe Li and Yijiang Chen and Colin Bergstrom and Ted Kim and Francesca Maria Olguin and Kelley Yuan and Matthew Abikenari and Andrew Heider and Sierra Willens and Sanjeeth Rajaram and Robert West and Joel Neal and Adam Schoenfeld and Maximilian Diehn and Chad Vanderbilt and Ruijiang Li},
year={2026},
eprint={2508.16085},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2508.16085},
}
```
|