|
Download README.md from PathOS-Code/artifact-aware-mil-prostate: direct link, hf CLI and curl.
- Browser
- Download file 7.13 kB
-
https://huggingface.co/PathOS-Code/artifact-aware-mil-prostate/resolve/main/README.md
- Command line
-
hf download hf://PathOS-Code/artifact-aware-mil-prostate/README.md
-
curl -L -o README.md https://huggingface.co/PathOS-Code/artifact-aware-mil-prostate/resolve/main/README.md
7.13 kB
| license: cc-by-nc-sa-4.0 | |
| tags: | |
| - pathology | |
| - computational-pathology | |
| - histopathology | |
| - multiple-instance-learning | |
| - whole-slide-images | |
| - prostate-cancer | |
| - medical-imaging | |
| - pytorch | |
| pipeline_tag: image-classification | |
| # Artifact-aware MIL heads for prostate biopsy ISUP grading | |
| Trained multiple instance learning (MIL) heads from the paper *Aggregation, | |
| Ordinal Training, and Artifact Gating in Multiple Instance Learning for | |
| Prostate Biopsy Grading*. Each model maps a bag of frozen Virchow2 patch | |
| embeddings from one prostate biopsy whole-slide image to an ISUP grade group | |
| (0–5). | |
| Code, data split and result files: <https://github.com/PathOS-Code/artifact-aware-mil-prostate> | |
| ## Contents | |
| 30 models, the members of every ensemble reported in the paper. Each folder | |
| holds `best_model.safetensors` (weights only), the run's recorded | |
| `config.json`, and `release_info.json` (best epoch, validation QWK, SHA-256). | |
| | Folder | Members | Aggregator | Loss | Gate | Protocol | | |
| | --- | --- | --- | --- | --- | --- | | |
| | `virchow2_transmil_focal_nogate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | none | fixed split, 5 seeds | | |
| | `virchow2_transmil_focal_maskgate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | mask-only | fixed split, 5 seeds | | |
| | `virchow2_transmil_focal_fullgate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | full robust | fixed split, 5 seeds | | |
| | `virchow2_abmil_focal_maskgate_5seed/` | `seed0`–`seed4` | ABMIL | focal | mask-only | fixed split, 5 seeds | | |
| | `virchow2_transmil_coral_maskgate_5fold/` | `fold_1`–`fold_5` | TransMIL-style, 2 layers | CORAL | mask-only | 5-fold CV | | |
| | `virchow2_transmil_coral_fullgate_5fold/` | `fold_1`–`fold_5` | TransMIL-style, 2 layers | CORAL | full robust | 5-fold CV | | |
| The mask-only and full robust gate variants implement the same model: flagged | |
| tokens are excluded from attention in both, so the attenuation factor α does | |
| not change the output. Their differences are run-to-run (replicate) | |
| differences. α was not logged for any run; the evaluation code uses the | |
| default α = 1e-4. | |
| Total size about 4.1 GB (TransMIL-style members 163 MB each, ABMIL members | |
| 6 MB each). The weights were exported from the training checkpoints with | |
| `scripts/training/export_weights.py` in the code repository; for every model | |
| the exported weights give identical logits to the original checkpoint. | |
| ## Input | |
| The models do **not** contain an image encoder. Each slide must be converted | |
| to a bag saved as a `.pt` file with: | |
| | Key | Shape | Content | | |
| | --- | --- | --- | | |
| | `embeddings` | `[N, 1280]` | Virchow2 CLS token of each tissue patch | | |
| | `spatial_variances` | `[N]` | variance of the Laplacian of the grayscale patch, used by the gate | | |
| | `coords` | `[N, 2]` | patch level-0 coordinates (used only for heatmaps) | | |
| Patches are 256 × 256 pixels at level 0 (nominal 20×), Macenko stain | |
| normalization with a fixed reference, resized to 224 × 224 for Virchow2. The | |
| code repository's `scripts/preprocessing/preprocess.py` and | |
| `scripts/feature_extraction/extract_features.py` produce these bags. | |
| **Virchow2 weights are not included.** Request access to | |
| [`paige-ai/Virchow2`](https://huggingface.co/paige-ai/Virchow2) and use it | |
| under its own license terms. | |
| ## Output | |
| - Focal-loss models: 6 logits (ISUP 0–5); class probabilities by softmax. | |
| - CORAL models: 5 ordinal threshold logits; class probabilities by | |
| differencing the cumulative sigmoids, clipping at 0 and renormalizing. | |
| Ensembles average the class probabilities of their members. | |
| ## Usage | |
| ```bash | |
| git clone https://github.com/PathOS-Code/artifact-aware-mil-prostate.git | |
| cd artifact-aware-mil-prostate | |
| uv sync | |
| hf download PathOS-Code/artifact-aware-mil-prostate --local-dir weights | |
| uv run python scripts/evaluation/ensemble_eval.py \ | |
| --exp-dirs weights/virchow2_transmil_focal_fullgate_5seed/seed{0,1,2,3,4} \ | |
| --embeddings-dir outputs/embeddings_virchow2 \ | |
| --labels-csv data/panda_labels.csv \ | |
| --split-csv splits/panda_fixed_split.csv \ | |
| --out outputs/ensembles/panda_focal_fullgate_5seed.json | |
| ``` | |
| For an external cohort, pass its embeddings and labels CSV with | |
| `--val-ratio 0 --test-ratio 0` to evaluate every slide. See `REPRODUCE.md` in | |
| the code repository for all experiments. | |
| ## Results | |
| From the paper and the result files in the code repository (`results/`). | |
| QWK = quadratic weighted kappa. | |
| PANDA held-out test split (1,061 slides; 10,614 slides overall from Radboud | |
| and Karolinska): | |
| | Ensemble | Test QWK | | |
| | --- | --- | | |
| | TransMIL-style, focal, no gate, 5 seeds | 0.9063 | | |
| | TransMIL-style, focal, mask-only, 5 seeds | 0.9104 | | |
| | TransMIL-style, focal, full robust gate, 5 seeds | 0.9134 (95% CI 0.8856–0.9368) | | |
| | ABMIL, focal, mask-only, 5 seeds | 0.8919 | | |
| PANDA five-fold cross-validation (out-of-fold QWK, optimistic): CORAL | |
| mask-only 0.9226, CORAL full robust gate 0.9227. | |
| External cohorts, direct transfer without fine-tuning: | |
| | Ensemble | SICAPv2 (155 biopsies) | CrowdGleason (232 slide identifiers) | | |
| | --- | --- | --- | | |
| | TransMIL-style, focal, no gate, 5 seeds | 0.8664 | 0.8076 | | |
| | TransMIL-style, focal, mask-only, 5 seeds | 0.8633 | 0.8028 | | |
| | TransMIL-style, focal, full robust gate, 5 seeds | 0.8482 | 0.8076 | | |
| | TransMIL-style, CORAL, mask-only, 5 folds | 0.8760 | 0.8109 | | |
| | TransMIL-style, CORAL, full robust gate, 5 folds | 0.8772 (95% CI 0.803–0.933) | 0.8080 (95% CI 0.733–0.871) | | |
| Caveats, as stated in the paper: | |
| - The PANDA test ensemble was the best of three ensembles, selected post hoc | |
| after exploratory runs that included the test slides. | |
| - None of the paired gated-versus-ungated QWK intervals excluded zero; no | |
| statistically detectable gating effect was found, and gate firing was not | |
| measured. | |
| - All intervals treat slides as independent and ignore patient clustering. | |
| - CrowdGleason slide labels are surrogates derived from patch labels. | |
| - Aggregator and ordinal-loss contrasts are confounded and descriptive. | |
| - Training is not bit-wise reproducible; retrained models differ slightly. | |
| ## Limitations | |
| - Developed on PANDA only (two institutions, routine-practice labels); | |
| external evaluation covers two small public cohorts. | |
| - Performance under other scanners, stains, tissue preparation or patient | |
| populations is unknown. | |
| - Each model predicts one ISUP grade group per slide. Attention heatmaps | |
| show which patches the model weighted, not where each Gleason pattern is; | |
| pattern-level segmentation is not part of this release. | |
| - Depends on Virchow2 embeddings; other encoders will not work with these | |
| weights. | |
| ## License | |
| The weights are released under | |
| [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) | |
| because they were trained on PANDA (CC BY-NC-SA 4.0): noncommercial use, | |
| attribution required, derivatives under the same license. The code in the | |
| GitHub repository is under the PolyForm Noncommercial License 1.0.0. | |
| ## Citation | |
| ```bibtex | |
| @article{TODO, | |
| title = {Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading}, | |
| author = {TODO}, | |
| journal = {TODO}, | |
| year = {TODO} | |
| } | |
| ``` |