docs: correct the reproduction instructions
#1
by Rohamzn - opened
DATA.md
CHANGED
|
@@ -37,8 +37,10 @@ idempotent. Verify before training:
|
|
| 37 |
python modeling/rebase_graph_paths.py --csv <csv> --check # expect N/N
|
| 38 |
```
|
| 39 |
|
| 40 |
-
|
| 41 |
-
|
|
|
|
|
|
|
| 42 |
|
| 43 |
## Graph format
|
| 44 |
|
|
@@ -93,20 +95,42 @@ python modeling/rebase_graph_paths.py --csv graphs/bach/extracted/sample_labels_
|
|
| 93 |
BRACS, 7-class lesion subtyping. 6 × ~1 GB tars under `bracs/data/` holding **only** `.pt`
|
| 94 |
files. Unlike BACH, the `sample_labels_rich.csv` sits one level up at `bracs/`.
|
| 95 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
### `graphs/breakhis/`: 709 graphs
|
| 97 |
BreakHis, binary benign/malignant. Small enough to ship as loose `.pt` files under
|
| 98 |
-
`breakhis/graphs/`, with `sample_labels_rich.csv` alongside them.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
|
| 100 |
### `graphs/spider_breast/`: 71,745 graphs
|
| 101 |
-
SPIDER-breast, 18 tumour-subtype classes, train 62,995 / test 8,750. 18 × ~
|
| 102 |
-
`spider_breast/data/` (`.pt` only; the
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
Two things to know:
|
| 105 |
|
| 106 |
- **The label files cover 92,892 patches, the graphs are 71,745.** Graph construction skips
|
| 107 |
any patch with fewer than 10 detected cells, which removes cell-sparse tissue almost
|
| 108 |
entirely: `Fat` retains 7 of 6,286 patches (1 in test), `Fibrosis` 415 of 6,260,
|
| 109 |
-
`Necrosis` 2,147 of 5,396,
|
|
|
|
|
|
|
| 110 |
matters, because `Fat` rests on a single test patch. `metadata.csv` is the file that matches the
|
| 111 |
graphs exactly.
|
| 112 |
- **`sample_id` is doubled in `metadata.csv`** (`patch_0000017_patch_0000017`, matching the
|
|
@@ -358,7 +382,9 @@ python studies/survival/investigate.py \
|
|
| 358 |
--input_csv survival.csv --output_dir out/survival --model_type graphist
|
| 359 |
```
|
| 360 |
|
| 361 |
-
Swap `--embeddings_dir` for
|
|
|
|
|
|
|
| 362 |
Requires `lifelines`.
|
| 363 |
|
| 364 |
**SPIDER-breast patch-level probe.** Same as slide-level but with `--scale patch` and
|
|
|
|
| 37 |
python modeling/rebase_graph_paths.py --csv <csv> --check # expect N/N
|
| 38 |
```
|
| 39 |
|
| 40 |
+
`--check` stops after 20,000 rows and says so (`(first 20k rows only)`), so it prints the
|
| 41 |
+
full count only for BACH (14,341) and BreakHis (709). For BRACS, SPIDER-breast and
|
| 42 |
+
TCGA-BRCA it prints `20000/20000`; that is the pass condition for those three. A
|
| 43 |
+
resolvable count below the total means graphs are being silently skipped.
|
| 44 |
|
| 45 |
## Graph format
|
| 46 |
|
|
|
|
| 95 |
BRACS, 7-class lesion subtyping. 6 × ~1 GB tars under `bracs/data/` holding **only** `.pt`
|
| 96 |
files. Unlike BACH, the `sample_labels_rich.csv` sits one level up at `bracs/`.
|
| 97 |
|
| 98 |
+
```bash
|
| 99 |
+
mkdir -p graphs/bracs/extracted
|
| 100 |
+
for t in graphs/bracs/data/*.tar; do tar -xf "$t" -C graphs/bracs/extracted; done
|
| 101 |
+
python modeling/rebase_graph_paths.py --csv graphs/bracs/sample_labels_rich.csv \
|
| 102 |
+
--mode absolute --root "$PWD/graphs/bracs/extracted"
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
### `graphs/breakhis/`: 709 graphs
|
| 106 |
BreakHis, binary benign/malignant. Small enough to ship as loose `.pt` files under
|
| 107 |
+
`breakhis/graphs/`, with `sample_labels_rich.csv` alongside them. Nothing to extract, but
|
| 108 |
+
the CSV still needs rebasing:
|
| 109 |
+
|
| 110 |
+
```bash
|
| 111 |
+
python modeling/rebase_graph_paths.py --csv graphs/breakhis/graphs/sample_labels_rich.csv \
|
| 112 |
+
--mode absolute --root "$PWD/graphs/breakhis/graphs"
|
| 113 |
+
```
|
| 114 |
|
| 115 |
### `graphs/spider_breast/`: 71,745 graphs
|
| 116 |
+
SPIDER-breast, 18 tumour-subtype classes, train 62,995 / test 8,750. 18 × ~255 MB tars under
|
| 117 |
+
`spider_breast/data/` (`.pt` only; the CSVs are one level up).
|
| 118 |
+
|
| 119 |
+
```bash
|
| 120 |
+
mkdir -p graphs/spider_breast/extracted
|
| 121 |
+
for t in graphs/spider_breast/data/*.tar; do tar -xf "$t" -C graphs/spider_breast/extracted; done
|
| 122 |
+
python modeling/rebase_graph_paths.py --csv graphs/spider_breast/sample_labels_rich.csv \
|
| 123 |
+
--mode absolute --root "$PWD/graphs/spider_breast/extracted"
|
| 124 |
+
```
|
| 125 |
|
| 126 |
Two things to know:
|
| 127 |
|
| 128 |
- **The label files cover 92,892 patches, the graphs are 71,745.** Graph construction skips
|
| 129 |
any patch with fewer than 10 detected cells, which removes cell-sparse tissue almost
|
| 130 |
entirely: `Fat` retains 7 of 6,286 patches (1 in test), `Fibrosis` 415 of 6,260,
|
| 131 |
+
`Necrosis` 2,147 of 5,396, `Lipogranuloma` 2,622 of 4,941 and `Vessels` 3,130 of 5,469.
|
| 132 |
+
Two more lose a tenth or less (`Benign phyllodes tumor` 89.9 %, `Fibroadenoma` 93.7 %);
|
| 133 |
+
the remaining eleven classes retain at least 97 %. In an 18-class macro-F1 this
|
| 134 |
matters, because `Fat` rests on a single test patch. `metadata.csv` is the file that matches the
|
| 135 |
graphs exactly.
|
| 136 |
- **`sample_id` is doubled in `metadata.csv`** (`patch_0000017_patch_0000017`, matching the
|
|
|
|
| 382 |
--input_csv survival.csv --output_dir out/survival --model_type graphist
|
| 383 |
```
|
| 384 |
|
| 385 |
+
Swap `--embeddings_dir` for `baselines/dinov2`, `baselines/mae` or
|
| 386 |
+
`baselines/graphist_v1/embeddings` to get the comparison rows. Note the extra level on the
|
| 387 |
+
last one: `baselines/graphist_v1/` itself holds the v1 checkpoint, not the embeddings.
|
| 388 |
Requires `lifelines`.
|
| 389 |
|
| 390 |
**SPIDER-breast patch-level probe.** Same as slide-level but with `--scale patch` and
|