File size: 7,134 Bytes
fa4163f
 
9baf699
afbfba6
 
 
 
 
 
 
 
 
fa4163f
9baf699
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8a8225e
 
 
9baf699
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
afbfba6
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
---
license: cc-by-nc-sa-4.0
tags:
- pathology
- computational-pathology
- histopathology
- multiple-instance-learning
- whole-slide-images
- prostate-cancer
- medical-imaging
- pytorch
pipeline_tag: image-classification
---

# Artifact-aware MIL heads for prostate biopsy ISUP grading

Trained multiple instance learning (MIL) heads from the paper *Aggregation,
Ordinal Training, and Artifact Gating in Multiple Instance Learning for
Prostate Biopsy Grading*. Each model maps a bag of frozen Virchow2 patch
embeddings from one prostate biopsy whole-slide image to an ISUP grade group
(0–5).

Code, data split and result files: <https://github.com/PathOS-Code/artifact-aware-mil-prostate>

## Contents

30 models, the members of every ensemble reported in the paper. Each folder
holds `best_model.safetensors` (weights only), the run's recorded
`config.json`, and `release_info.json` (best epoch, validation QWK, SHA-256).

| Folder | Members | Aggregator | Loss | Gate | Protocol |
| --- | --- | --- | --- | --- | --- |
| `virchow2_transmil_focal_nogate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | none | fixed split, 5 seeds |
| `virchow2_transmil_focal_maskgate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | mask-only | fixed split, 5 seeds |
| `virchow2_transmil_focal_fullgate_5seed/` | `seed0`–`seed4` | TransMIL-style, 2 layers | focal | full robust | fixed split, 5 seeds |
| `virchow2_abmil_focal_maskgate_5seed/` | `seed0`–`seed4` | ABMIL | focal | mask-only | fixed split, 5 seeds |
| `virchow2_transmil_coral_maskgate_5fold/` | `fold_1`–`fold_5` | TransMIL-style, 2 layers | CORAL | mask-only | 5-fold CV |
| `virchow2_transmil_coral_fullgate_5fold/` | `fold_1`–`fold_5` | TransMIL-style, 2 layers | CORAL | full robust | 5-fold CV |

The mask-only and full robust gate variants implement the same model: flagged
tokens are excluded from attention in both, so the attenuation factor α does
not change the output. Their differences are run-to-run (replicate)
differences. α was not logged for any run; the evaluation code uses the
default α = 1e-4.

Total size about 4.1 GB (TransMIL-style members 163 MB each, ABMIL members
6 MB each). The weights were exported from the training checkpoints with
`scripts/training/export_weights.py` in the code repository; for every model
the exported weights give identical logits to the original checkpoint.

## Input

The models do **not** contain an image encoder. Each slide must be converted
to a bag saved as a `.pt` file with:

| Key | Shape | Content |
| --- | --- | --- |
| `embeddings` | `[N, 1280]` | Virchow2 CLS token of each tissue patch |
| `spatial_variances` | `[N]` | variance of the Laplacian of the grayscale patch, used by the gate |
| `coords` | `[N, 2]` | patch level-0 coordinates (used only for heatmaps) |

Patches are 256 × 256 pixels at level 0 (nominal 20×), Macenko stain
normalization with a fixed reference, resized to 224 × 224 for Virchow2. The
code repository's `scripts/preprocessing/preprocess.py` and
`scripts/feature_extraction/extract_features.py` produce these bags.

**Virchow2 weights are not included.** Request access to
[`paige-ai/Virchow2`](https://huggingface.co/paige-ai/Virchow2) and use it
under its own license terms.

## Output

- Focal-loss models: 6 logits (ISUP 0–5); class probabilities by softmax.
- CORAL models: 5 ordinal threshold logits; class probabilities by
  differencing the cumulative sigmoids, clipping at 0 and renormalizing.

Ensembles average the class probabilities of their members.

## Usage

```bash
git clone https://github.com/PathOS-Code/artifact-aware-mil-prostate.git
cd artifact-aware-mil-prostate
uv sync
hf download PathOS-Code/artifact-aware-mil-prostate --local-dir weights

uv run python scripts/evaluation/ensemble_eval.py \
  --exp-dirs weights/virchow2_transmil_focal_fullgate_5seed/seed{0,1,2,3,4} \
  --embeddings-dir outputs/embeddings_virchow2 \
  --labels-csv data/panda_labels.csv \
  --split-csv splits/panda_fixed_split.csv \
  --out outputs/ensembles/panda_focal_fullgate_5seed.json
```

For an external cohort, pass its embeddings and labels CSV with
`--val-ratio 0 --test-ratio 0` to evaluate every slide. See `REPRODUCE.md` in
the code repository for all experiments.

## Results

From the paper and the result files in the code repository (`results/`).
QWK = quadratic weighted kappa.

PANDA held-out test split (1,061 slides; 10,614 slides overall from Radboud
and Karolinska):

| Ensemble | Test QWK |
| --- | --- |
| TransMIL-style, focal, no gate, 5 seeds | 0.9063 |
| TransMIL-style, focal, mask-only, 5 seeds | 0.9104 |
| TransMIL-style, focal, full robust gate, 5 seeds | 0.9134 (95% CI 0.8856–0.9368) |
| ABMIL, focal, mask-only, 5 seeds | 0.8919 |

PANDA five-fold cross-validation (out-of-fold QWK, optimistic): CORAL
mask-only 0.9226, CORAL full robust gate 0.9227.

External cohorts, direct transfer without fine-tuning:

| Ensemble | SICAPv2 (155 biopsies) | CrowdGleason (232 slide identifiers) |
| --- | --- | --- |
| TransMIL-style, focal, no gate, 5 seeds | 0.8664 | 0.8076 |
| TransMIL-style, focal, mask-only, 5 seeds | 0.8633 | 0.8028 |
| TransMIL-style, focal, full robust gate, 5 seeds | 0.8482 | 0.8076 |
| TransMIL-style, CORAL, mask-only, 5 folds | 0.8760 | 0.8109 |
| TransMIL-style, CORAL, full robust gate, 5 folds | 0.8772 (95% CI 0.803–0.933) | 0.8080 (95% CI 0.733–0.871) |

Caveats, as stated in the paper:

- The PANDA test ensemble was the best of three ensembles, selected post hoc
  after exploratory runs that included the test slides.
- None of the paired gated-versus-ungated QWK intervals excluded zero; no
  statistically detectable gating effect was found, and gate firing was not
  measured.
- All intervals treat slides as independent and ignore patient clustering.
- CrowdGleason slide labels are surrogates derived from patch labels.
- Aggregator and ordinal-loss contrasts are confounded and descriptive.
- Training is not bit-wise reproducible; retrained models differ slightly.

## Limitations

- Developed on PANDA only (two institutions, routine-practice labels);
  external evaluation covers two small public cohorts.
- Performance under other scanners, stains, tissue preparation or patient
  populations is unknown.
- Each model predicts one ISUP grade group per slide. Attention heatmaps
  show which patches the model weighted, not where each Gleason pattern is;
  pattern-level segmentation is not part of this release.
- Depends on Virchow2 embeddings; other encoders will not work with these
  weights.

## License

The weights are released under
[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)
because they were trained on PANDA (CC BY-NC-SA 4.0): noncommercial use,
attribution required, derivatives under the same license. The code in the
GitHub repository is under the PolyForm Noncommercial License 1.0.0.

## Citation

```bibtex
@article{TODO,
  title   = {Aggregation, Ordinal Training, and Artifact Gating in Multiple Instance Learning for Prostate Biopsy Grading},
  author  = {TODO},
  journal = {TODO},
  year    = {TODO}
}
```