| --- |
| license: cc-by-nc-sa-4.0 |
| library_name: pytorch |
| pipeline_tag: graph-ml |
| tags: |
| - graph-neural-networks |
| - histopathology |
| - self-supervised-learning |
| - pytorch-geometric |
| - graph-representation-learning |
| - edge-features |
| --- |
| |
| # GrapHist++: Edge-Informed Graph Self-Supervised Learning for Histopathology |
|
|
| GrapHist++ encodes a histopathology tile as a **cell graph** (one node per nucleus, edges |
| from a Delaunay triangulation pruned at 100 µm) and embeds it with a self-supervised |
| **ACM-GINEConv** encoder. It extends [GrapHist](https://huggingface.co/ogutsevda/graphist) |
| ([arXiv:2603.00143](https://arxiv.org/abs/2603.00143)) by making the **edges carry |
| information**: each edge holds a 75-dimensional descriptor of the tissue between two cells, |
| and the encoder injects those features into message passing instead of treating edges as mere |
| connectivity. |
|
|
| **Trained on 11,149,499 cell graphs from TCGA-BRCA**, 100 epochs, batch 2048, no labels. The |
| objective is GraphMAE-style masked node-feature reconstruction (mask 50 %, scaled cosine |
| error, α = 3) plus a **VICReg** variance/covariance term on the pooled graph embedding, which |
| keeps all embedding dimensions in use (final `pca_1` 0.17, effective dimension 11.9). |
|
|
| Encoder 7.98 M parameters (9.29 M on the inference path, which also runs the projection; |
| 10.53 M for the full pre-training model), 512-dim embeddings, 0.093 ms/patch at batch 48 on an |
| H200. |
|
|
| ## Results |
|
|
| Slide-level MIL, transfer to unseen cohorts (test macro-F1 %, best of three MIL heads): |
|
|
| | | BRACS | BreakHis | BACH | |
| |---|:--:|:--:|:--:| |
| | GrapHist (v1) | 60.30 | 89.37 | **69.16** | |
| | **GrapHist++** | **69.00** | **95.53** | 68.98 | |
|
|
| Survival on TCGA-BRCA (Cox PH, C-index): **0.793**, vs 0.763 for v1, 0.724 MAE, 0.632 DINOv2. |
| Cell-type identification and patch-level subtyping are in [`DATA.md`](DATA.md), which maps |
| every artifact here to the result it reproduces. |
|
|
| ## What's in this repo |
|
|
| ``` |
| graphist_v2.pt the model. 126,597,010 B, md5 81a2e6b91cefff0bbbc13c6fd318ee78 |
| modeling/ code to load and run it, plus the graph-path helper |
| graphs/ cell graphs: TCGA-BRCA (11.1 M, pre-training), BACH, BRACS, |
| BreakHis, SPIDER-breast |
| embeddings/ embeddings from this model: slide-level (4 cohorts), cell-level (22 sets) |
| baselines/ DINOv2, MAE and GrapHist v1 features for the comparison rows |
| labels/ TCGA-BRCA clinical table and slide labels |
| studies/ data for the side analyses: homophily, AdapterGNN, preprocessing runtime |
| upstream_v1/ mirror of the original GrapHist v1 releases |
| ``` |
|
|
| To run the model on your own graphs you need only `graphist_v2.pt` and `modeling/`. To |
| reproduce a published number you also need that dataset's `normalization.json` from |
| `graphs/<cohort>/`, since the transforms are part of the pipeline. See [`DATA.md`](DATA.md). |
|
|
| Training, evaluation and analysis code lives in |
| [github.com/Ace3Z/GrapHist-V2](https://github.com/Ace3Z/GrapHist-V2). |
|
|
| > ⚠️ **If you use `graphs/`, run `modeling/rebase_graph_paths.py` first.** The label CSVs store |
| > bare filenames and the loader resolves them against the working directory, so without this |
| > step every graph is silently dropped. `DATA.md` has the commands and the expected counts. |
|
|
| ## Usage |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| path = snapshot_download(repo_id="Ace3Z/graphist-v2") |
| ``` |
|
|
| Three arguments are **load-critical**. `build_model` reads two of them with defaults that do |
| not match this checkpoint, and the third has no default at all: |
|
|
| | Argument | Must be | If wrong | |
| |---|---|---| |
| | `edge_distance_in_proj` | `False` | `edge_input_proj` built as (512, **75**); checkpoint has (512, **74**) | |
| | `encoder_norm` | `"layer"` | `encoder.layer_norms.*` missing from the model | |
| | `concat_hidden` | `True` | `AttributeError` if omitted; (512, 512) instead of (512, **2560**) if `False` | |
|
|
| ```python |
| import sys, torch |
| sys.path.insert(0, f"{path}/modeling") |
| from models import build_model |
| |
| class Args: |
| encoder = decoder = "acm_gineconv" |
| num_features = 96 # per-cell features |
| num_edge_features = 75 # per-edge features (projection sees 74; distance excluded) |
| num_hidden = 512 |
| num_layers = 5 |
| concat_hidden = True # load-critical |
| encoder_norm = "layer" # load-critical |
| edge_distance_in_proj = False # load-critical |
| input_norm = "none" |
| batchnorm = False |
| activation = "prelu" |
| loss_fn = "sce" |
| alpha_l = 3 |
| mask_rate = 0.5 |
| replace_rate = 0.1 |
| drop_edge_rate = 0.0 |
| vicreg_var_weight = 0.05 # training only |
| vicreg_cov_weight = 0.002 |
| vicreg_gamma = 1.0 |
| |
| model = build_model(Args()) |
| ckpt = torch.load(f"{path}/graphist_v2.pt", map_location="cpu", weights_only=False) |
| model.load_state_dict(ckpt["model_state_dict"], strict=True) # note the key name |
| model.eval() |
| |
| # x: [num_nodes, 96] edge_index: [2, num_edges] edge_attr: [num_edges, 75] |
| # batch: [num_nodes] graph assignment (zeros for a single graph) |
| with torch.no_grad(): |
| node_emb = model.embed(x, edge_index, edge_attr, batch) # -> [num_nodes, 512] |
| ``` |
|
|
| Region- and slide-level embeddings are means over node embeddings. To match the published |
| numbers, apply the transforms used at training time: `NormalizeData` with the dataset's |
| `normalization.json`, then `AddVirtualNode`. Both are in `modeling/graphist_utils.py`. |
|
|
| ### Requirements |
|
|
| Inference needs only `torch`, `torch_geometric`, `numpy` and `pandas`. |
|
|
| | | Trained with | Also verified on | |
| |---|---|---| |
| | Python | 3.10 | 3.10 | |
| | torch | 2.2.2 (CUDA 11.8) | 2.10.0 (CUDA 12.8) | |
| | torch_geometric | 2.5.2 | 2.7.0 | |
| | numpy | 1.26.4 | 2.2.6 | |
| | pandas | 2.2.2 | 2.3.3 | |
| |
| Tested at both ends of that range, and on both numpy 1.x and 2.x. Newer versions also work; |
| these are floors, not a supported ceiling. CPU-only inference works. |
| |
| Using the released data needs a little more: `h5py` (the embedding files are HDF5), |
| `scikit-learn` 1.5+ for the cell-level probe, and `lifelines` 0.30 for the survival analysis. |
| |
| > `modeling/models/acm_gin.py` is required even though this model never uses it: |
| > `edcoder.py` imports it at module level. |
|
|
| ## Graph format |
|
|
| | Tensor | Shape | Meaning | |
| |---|---|---| |
| | `x` | `[num_nodes, 96]` | per-cell morphology, texture and colour features | |
| | `edge_index` | `[2, num_edges]` | Delaunay edges, pruned at 100 µm | |
| | `edge_attr` | `[num_edges, 75]` | column 0 = centroid distance (µm); 1–74 describe the inter-cellular region | |
|
|
| The v1 releases use a 1-dimensional `edge_attr` (distance only) and are **not** interchangeable |
| with this model in either direction. |
|
|
| ## Verification |
|
|
| `build_model` + `load_state_dict(strict=True)` returns 0 missing / 0 unexpected keys using only |
| the files shipped here. All 397 BACH slide embeddings regenerate from this checkpoint at |
| minimum cosine similarity **0.9999999997** against the published set. As a control, a |
| different checkpoint scores 0.256 mean cosine on the same comparison, so the test discriminates |
| and these weights are the ones behind the released embeddings. |
|
|
| ## Datasets used |
|
|
| This release is built from seven public cohorts. If you use it, please cite GrapHist **and** |
| the source cohort(s) your work touches. |
|
|
| | Cohort | Used for | Source | |
| |---|---|---| |
| | **TCGA-BRCA** | pre-training (11.1 M graphs), slide-level evaluation, survival | [GDC Data Portal](https://portal.gdc.cancer.gov/) | |
| | **BACH** (ICIAR 2018) | slide-level subtyping, homophily | [Grand Challenge](https://iciar2018-challenge.grand-challenge.org/) | |
| | **BRACS** | slide-level subtyping, homophily | [bracs.icar.cnr.it](https://www.bracs.icar.cnr.it/) | |
| | **BreakHis** | slide-level subtyping, homophily | [P&D Lab, UFPR](https://web.inf.ufpr.br/vri/databases/breast-cancer-histopathological-database-breakhis/) | |
| | **NuCLS** | cell-type identification, homophily | [NuCLS](https://sites.google.com/view/nucls/home) | |
| | **PanNuke** | cell-type identification, homophily | [TIA Centre, Warwick](https://warwick.ac.uk/fac/sci/dcs/research/tia/data/pannuke/) | |
| | **SPIDER-breast** | patch-level subtyping | [histai/SPIDER-breast](https://huggingface.co/datasets/histai/SPIDER-breast) | |
|
|
| <details> |
| <summary>BibTeX for the source cohorts</summary> |
|
|
| ```bibtex |
| @article{weinstein2013cancer, |
| title={The cancer genome atlas pan-cancer analysis project}, |
| author={Weinstein, John N and Collisson, Eric A and Mills, Gordon B and Shaw, Kenna R and |
| Ozenberger, Brad A and Ellrott, Kyle and Shmulevich, Ilya and Sander, Chris and |
| Stuart, Joshua M}, |
| journal={Nature Genetics}, volume={45}, number={10}, pages={1113--1120}, year={2013}, |
| publisher={Nature Publishing Group} |
| } |
| |
| @article{aresta2019bach, |
| title={{BACH}: Grand challenge on breast cancer histology images}, |
| author={Aresta, Guilherme and Ara{\'u}jo, Teresa and Kwok, Scotty and |
| Chennamsetty, Sai Saketh and Safwan, Mohammed and Alex, Varghese and others}, |
| journal={Medical Image Analysis}, volume={56}, pages={122--139}, year={2019}, |
| publisher={Elsevier} |
| } |
| |
| @article{brancati2022bracs, |
| title={{BRACS}: A Dataset for BReAst Carcinoma Subtyping in {H\&E} Histology Images}, |
| author={Brancati, Nadia and Anniciello, Anna Maria and Pati, Pushpak and Riccio, Daniel and |
| Scognamiglio, Giosu{\`e} and Jaume, Guillaume and De Pietro, Giuseppe and |
| Di Bonito, Maurizio and Foncubierta, Antonio and Botti, Gerardo and others}, |
| journal={Database}, volume={2022}, pages={baac093}, year={2022}, |
| publisher={Oxford University Press UK} |
| } |
| |
| @article{spanhol2015dataset, |
| title={A dataset for breast cancer histopathological image classification}, |
| author={Spanhol, Fabio A and Oliveira, Luiz S and Petitjean, Caroline and Heutte, Laurent}, |
| journal={IEEE Transactions on Biomedical Engineering}, volume={63}, number={7}, |
| pages={1455--1462}, year={2015}, publisher={IEEE} |
| } |
| |
| @article{amgad2022nucls, |
| title={{NuCLS}: A scalable crowdsourcing approach and dataset for nucleus classification and |
| segmentation in breast cancer}, |
| author={Amgad, Mohamed and Atteya, Lamees A and Hussein, Hagar and Mohammed, Kareem Hosny and |
| Hafiz, Ehab and Elsebaie, Maha AT and Alhusseiny, Ahmed M and |
| AlMoslemany, Mohamed Atef and Elmatboly, Abdelmagid M and Pappalardo, Philip A and others}, |
| journal={GigaScience}, volume={11}, pages={giac037}, year={2022}, |
| publisher={Oxford University Press} |
| } |
| |
| @article{gamper2020pannuke, |
| title={{PanNuke} dataset extension, insights and baselines}, |
| author={Gamper, Jevgenij and Koohbanani, Navid Alemi and Benes, Ksenija and Graham, Simon and |
| Jahanifar, Mostafa and Khurram, Syed Ali and Azam, Ayesha and Hewitt, Katherine and |
| Rajpoot, Nasir}, |
| journal={arXiv preprint arXiv:2003.10778}, year={2020} |
| } |
| |
| @article{nechaev2025spider, |
| title={{SPIDER}: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Models}, |
| author={Nechaev, Dmitry and Pchelnikov, Alexey and Ivanova, Ekaterina}, |
| year={2025}, eprint={2503.02876}, archivePrefix={arXiv}, primaryClass={cs.CV} |
| } |
| ``` |
| </details> |
|
|
| ## Licence and citation |
|
|
| Released `cc-by-nc-sa-4.0`. Everything here is *derived cell-level features*, not images. |
| SPIDER-breast is `cc-by-nc-4.0`, research use only, and its terms travel with the derived |
| graphs. `labels/tcga_brca_clinical.tsv` is the open-access GDC clinical export, redistributed |
| under TCGA's open-access terms. The source cohorts keep their own licences. |
|
|
| ```bibtex |
| @article{ogut2026graphist, |
| title = {GrapHist: Graph Self-Supervised Learning for Histopathology}, |
| author = {{\"O}{\u{g}}{\"u}t, Sevda and Vincent-Cuaz, C{\'e}dric and |
| Dubljevic, Natalia and Hurtado, Carlos and Subramanian, Vaishnavi and |
| Frossard, Pascal and Thanou, Dorina}, |
| year = {2026}, |
| eprint = {2603.00143}, |
| archivePrefix = {arXiv}, |
| primaryClass = {cs.CV} |
| } |
| ``` |
|
|
| Built on GraphMAE (Hou et al., 2022), ACM (Luan et al., 2022), GINEConv (Hu et al., 2020) and |
| VICReg (Bardes et al., 2022). |
|
|
| This work was done by [**Mahbod Tajdini**](https://mahbodtajdini.com) and [**Tomás Gadea Alcaide**](https://tomasgadea.com/), supervised by members of |
| [LTS4, EPFL](https://www.epfl.ch/labs/lts4/). |
|
|
| Training, evaluation and analysis code: |
| [github.com/Ace3Z/GrapHist-V2](https://github.com/Ace3Z/GrapHist-V2). This repository holds the |
| model, the code needed to load it, and the data. |
|
|
