docs: correct the reproduction instructions

#1
by Rohamzn - opened
Files changed (1) hide show
  1. DATA.md +33 -7
DATA.md CHANGED
@@ -37,8 +37,10 @@ idempotent. Verify before training:
37
  python modeling/rebase_graph_paths.py --csv <csv> --check # expect N/N
38
  ```
39
 
40
- Expected: BACH 14,341, BreakHis 709, BRACS 96,153, SPIDER-breast 71,745,
41
- TCGA-BRCA 11,149,499. Anything less means graphs are being silently skipped.
 
 
42
 
43
  ## Graph format
44
 
@@ -93,20 +95,42 @@ python modeling/rebase_graph_paths.py --csv graphs/bach/extracted/sample_labels_
93
  BRACS, 7-class lesion subtyping. 6 × ~1 GB tars under `bracs/data/` holding **only** `.pt`
94
  files. Unlike BACH, the `sample_labels_rich.csv` sits one level up at `bracs/`.
95
 
 
 
 
 
 
 
 
96
  ### `graphs/breakhis/`: 709 graphs
97
  BreakHis, binary benign/malignant. Small enough to ship as loose `.pt` files under
98
- `breakhis/graphs/`, with `sample_labels_rich.csv` alongside them.
 
 
 
 
 
 
99
 
100
  ### `graphs/spider_breast/`: 71,745 graphs
101
- SPIDER-breast, 18 tumour-subtype classes, train 62,995 / test 8,750. 18 × ~245 MB tars under
102
- `spider_breast/data/` (`.pt` only; the CSV is one level up).
 
 
 
 
 
 
 
103
 
104
  Two things to know:
105
 
106
  - **The label files cover 92,892 patches, the graphs are 71,745.** Graph construction skips
107
  any patch with fewer than 10 detected cells, which removes cell-sparse tissue almost
108
  entirely: `Fat` retains 7 of 6,286 patches (1 in test), `Fibrosis` 415 of 6,260,
109
- `Necrosis` 2,147 of 5,396, while ten other classes lose ~0 %. In an 18-class macro-F1 this
 
 
110
  matters, because `Fat` rests on a single test patch. `metadata.csv` is the file that matches the
111
  graphs exactly.
112
  - **`sample_id` is doubled in `metadata.csv`** (`patch_0000017_patch_0000017`, matching the
@@ -358,7 +382,9 @@ python studies/survival/investigate.py \
358
  --input_csv survival.csv --output_dir out/survival --model_type graphist
359
  ```
360
 
361
- Swap `--embeddings_dir` for any directory under `baselines/` to get the comparison rows.
 
 
362
  Requires `lifelines`.
363
 
364
  **SPIDER-breast patch-level probe.** Same as slide-level but with `--scale patch` and
 
37
  python modeling/rebase_graph_paths.py --csv <csv> --check # expect N/N
38
  ```
39
 
40
+ `--check` stops after 20,000 rows and says so (`(first 20k rows only)`), so it prints the
41
+ full count only for BACH (14,341) and BreakHis (709). For BRACS, SPIDER-breast and
42
+ TCGA-BRCA it prints `20000/20000`; that is the pass condition for those three. A
43
+ resolvable count below the total means graphs are being silently skipped.
44
 
45
  ## Graph format
46
 
 
95
  BRACS, 7-class lesion subtyping. 6 × ~1 GB tars under `bracs/data/` holding **only** `.pt`
96
  files. Unlike BACH, the `sample_labels_rich.csv` sits one level up at `bracs/`.
97
 
98
+ ```bash
99
+ mkdir -p graphs/bracs/extracted
100
+ for t in graphs/bracs/data/*.tar; do tar -xf "$t" -C graphs/bracs/extracted; done
101
+ python modeling/rebase_graph_paths.py --csv graphs/bracs/sample_labels_rich.csv \
102
+ --mode absolute --root "$PWD/graphs/bracs/extracted"
103
+ ```
104
+
105
  ### `graphs/breakhis/`: 709 graphs
106
  BreakHis, binary benign/malignant. Small enough to ship as loose `.pt` files under
107
+ `breakhis/graphs/`, with `sample_labels_rich.csv` alongside them. Nothing to extract, but
108
+ the CSV still needs rebasing:
109
+
110
+ ```bash
111
+ python modeling/rebase_graph_paths.py --csv graphs/breakhis/graphs/sample_labels_rich.csv \
112
+ --mode absolute --root "$PWD/graphs/breakhis/graphs"
113
+ ```
114
 
115
  ### `graphs/spider_breast/`: 71,745 graphs
116
+ SPIDER-breast, 18 tumour-subtype classes, train 62,995 / test 8,750. 18 × ~255 MB tars under
117
+ `spider_breast/data/` (`.pt` only; the CSVs are one level up).
118
+
119
+ ```bash
120
+ mkdir -p graphs/spider_breast/extracted
121
+ for t in graphs/spider_breast/data/*.tar; do tar -xf "$t" -C graphs/spider_breast/extracted; done
122
+ python modeling/rebase_graph_paths.py --csv graphs/spider_breast/sample_labels_rich.csv \
123
+ --mode absolute --root "$PWD/graphs/spider_breast/extracted"
124
+ ```
125
 
126
  Two things to know:
127
 
128
  - **The label files cover 92,892 patches, the graphs are 71,745.** Graph construction skips
129
  any patch with fewer than 10 detected cells, which removes cell-sparse tissue almost
130
  entirely: `Fat` retains 7 of 6,286 patches (1 in test), `Fibrosis` 415 of 6,260,
131
+ `Necrosis` 2,147 of 5,396, `Lipogranuloma` 2,622 of 4,941 and `Vessels` 3,130 of 5,469.
132
+ Two more lose a tenth or less (`Benign phyllodes tumor` 89.9 %, `Fibroadenoma` 93.7 %);
133
+ the remaining eleven classes retain at least 97 %. In an 18-class macro-F1 this
134
  matters, because `Fat` rests on a single test patch. `metadata.csv` is the file that matches the
135
  graphs exactly.
136
  - **`sample_id` is doubled in `metadata.csv`** (`patch_0000017_patch_0000017`, matching the
 
382
  --input_csv survival.csv --output_dir out/survival --model_type graphist
383
  ```
384
 
385
+ Swap `--embeddings_dir` for `baselines/dinov2`, `baselines/mae` or
386
+ `baselines/graphist_v1/embeddings` to get the comparison rows. Note the extra level on the
387
+ last one: `baselines/graphist_v1/` itself holds the v1 checkpoint, not the embeddings.
388
  Requires `lifelines`.
389
 
390
  **SPIDER-breast patch-level probe.** Same as slide-level but with `--scale patch` and