Document BRACS graphs + v2 embeddings: purpose, consuming branch and paper table for every artifact

data/README.md

@@ -6,17 +6,22 @@ branch's results it produces**. Branch names refer to
 Table numbers refer to `results_for_writing_LOG/RESULTS_OVERVIEW.md` on the
 `gineconv_edge_updates` branch.
 
-## Uploaded so far
+## Uploaded so far — what each artifact is for, and which branch's results it produces
 
-| Path | What it is | What it is FOR | Branch | Result |
+| Path | What it is | What it is FOR | Branch | Paper result |
 |---|---|---|---|---|
-| `v2_graphs/bach/` | 14,341 cell graphs, BACH (ICIAR 2018), 224 px tiles at 20× | Input to `generate_embs.py --scale slide` → slide embeddings → attention-MIL subtyping (4-class: Normal / Benign / InSitu / Invasive). **Also the fixed dataset of the Table 4 inference profile** (`run_profile_table4.sh`, batch 48) | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 2** row *BACH*; **Table 4** inference row |
-| `v2_graphs/breakhis/` | 709 cell graphs, BreakHis, 224 px tiles at 20× | Same path; binary benign vs malignant subtyping | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 2**, row *BreakHis* |
-
-Reported macro-F1 for these two rows (v2 / v2+VICReg / vanilla GrapHist):
-BACH 66.17 ± 3.10 / 64.20 ± 3.15 / **69.16 ± 3.37** · BreakHis 92.65 ± 1.01 /
-**95.53 ± 2.44** / 89.37 ± 1.94. BreakHis is where VICReg helps most on transfer (+6.2 over
-vanilla); BACH is the one dataset where vanilla still leads, within ~1σ.
+| `v2_graphs/bach/` | 14,341 cell graphs, BACH (ICIAR 2018), 224 px @ 20×, 6 tars→1 tar | `generate_embs.py --scale slide` → slide embeddings → attention-MIL, 4-class (Normal/Benign/InSitu/Invasive). **Also the fixed dataset of the Table 4 inference profile** (`run_profile_table4.sh`, batch 48) | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 2** row *BACH*; **Table 4** inference row |
+| `v2_graphs/breakhis/` | 709 cell graphs, BreakHis, loose `.pt` | same path; binary benign vs malignant | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 2** row *BreakHis* |
+| `v2_graphs/bracs/` | 96,153 cell graphs, BRACS, 6 × ~1 GB tars | same path; 7-class lesion subtyping | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 2** row *BRACS* |
+| `v2_embeddings/slide/` | per-slide `embeddings.h5`, `(n_tiles, 512)` — BACH 397 · BRACS 4,493 · BreakHis 522 · TCGA-BRCA 1,119 | (a) `run_mil_slide.sh` / `main_slide.py`: 5-fold MIL × 3 heads. (b) `src/survival_analysis/prepare_dataframe.py`: tiles→slide→patient mean-pool → Cox PH + Kaplan-Meier | `gineconv_edge_updates` (MIL); **`main`** (survival) | **Table 2** (+VICReg) and **Table 3** (+VICReg row, C-index 0.793) |
+| `v2_embeddings/cell/` | 22 tars of per-graph `.npz` (keys `embedding`, `label`): `NuCLS_{main,super}_fold1-5`, `PanNuke_{20x,40x}_test1-3`, `PanNuke_breast_{20x,40x}_test1-3` | `main_cell.py`: StandardScaler + multinomial logistic regression (`class_weight=balanced`, `max_iter=2000`, seed 0) | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | **Table 5** (+VICReg rows) — **the only reproducible path**, see the cell-level note below |
+| `graphist_V2.pt` (repo root) | the encoder these all run through | frozen `--checkpoint_path` for every v2 embedding run | `gineconv_edge_updates`, `gineconv-edge-updates-clean` | Tables 2–5 (+VICReg), Table 4 params |
+
+Reported macro-F1 (v2 / v2+VICReg / vanilla GrapHist): BACH 66.17 ± 3.10 / 64.20 ± 3.15 /
+**69.16 ± 3.37** · BreakHis 92.65 ± 1.01 / **95.53 ± 2.44** / 89.37 ± 1.94 · BRACS
+63.62 ± 1.30 / **69.00 ± 1.30** / 60.30 ± 0.46. VICReg's gains are a **transfer** effect
+(BRACS +8.7, BreakHis +6.2 over vanilla); BACH is the one dataset where vanilla still leads,
+within ~1σ.
 
 ## Graph format — these are the **v2 (75-dim edge)** datasets
 
@@ -103,10 +108,8 @@ embeddings. The `--edge_distance_in_proj False`, `--encoder_norm layer` and
 
 | Planned path | Contents | Branch | Result |
 |---|---|---|---|
-| `v2_graphs/bracs/` | 96,153 graphs | `gineconv_edge_updates` | Table 2, row *BRACS* |
 | `v2_graphs/tcga_brca/` | 11,149,499 graphs, 647 GB — the pre-training corpus | `gineconv_edge_updates` | pre-training; Table 2 row *TCGA-BRCA (ID)* |
 | `v2_graphs/spider_breast/` | 71,745 graphs — **on hold**: `sample_split.csv` and `sample_labels.csv` each carry 92,892 rows against 71,745 graphs, unresolved | (new work) | backs no table |
-| `v2_embeddings/` | 26 dirs, 29.8 GB | `main`, `gineconv_edge_updates` | **Tables 3 & 5** |
 | `v1_graphs/{nucls,pannuke}/` | 3,388 + 14,414 graphs, 1-dim edges | `AdapterGNN` | Table 5 (AdapterGNN reproduction) |
 | `homophily_tiles/` | 109,904 tile graphs + CellViT pseudo-labels | `homophily-heterophily-metrics` | homophily / fat-tail appendix |
 | `baseline_embeddings/` | DINOv2 + MAE + GrapHist-v1 TCGA-BRCA | `batch_effects` | batch-effect analysis |