README.md

metadata

viewer: true
license: cc-by-nc-sa-4.0
language:
  - en
tags:
  - spatial-transcriptomics
  - histology
  - pathology
  - benchmark
task_categories:
  - feature-extraction
extra_gated_prompt: >-
  - This model and associated code are released under the [CC-BY-NC-ND 4.0
  license](https://creativecommons.org/licenses/by-nc-nd/4.0/) and may only be
  used for non-commercial, academic research purposes with proper attribution. 
  - Any commercial use, sale, or other monetization of the model and its
  derivatives, is prohibited and requires prior approval.  - By downloading the
  model, you attest that all information (affiliation, research use) is correct
  and up-to-date. Downloading the model requires prior registration on Hugging
  Face and agreeing to the terms of use. By downloading this model, you agree
  not to distribute, publish or reproduce a copy of the model. If another user
  within your organization wishes to use the model, they must register as an
  individual user and agree to comply with the terms of use.  - This model is
  provided “as-is” without warranties of any kind, express or implied. This
  model has not been reviewed, certified, or approved by any regulatory body,
  including but not limited to the FDA (U.S.), EMA (Europe), MHRA (UK), or other
  medical device authorities. Any application of this model in healthcare or
  biomedical settings must comply with relevant regulatory requirements and
  undergo independent validation. Users assume full responsibility for how they
  use this model and any resulting consequences. The authors, contributors, and
  distributors disclaim any liability for damages, direct or indirect, resulting
  from model use. Users are responsible for ensuring compliance with data
  protection regulations (e.g., GDPR, HIPAA) when using it in research that
  involves patient data.
extra_gated_fields:
  Full Name (first and last): text
  Type of Affiliation:
    type: select
    options:
      - Industry
      - Academia
      - Other
  Current Affiliation (no abbreviations): text
  Current and Official Institutional Email: text
  Main use-case:
    type: select
    options:
      - Models Benchmarking
      - Biomarker Discovery
      - Diagnostics
      - Pathology Workflows Acceleration
      - Other
  Please add information on your intended research use: text
  I agree to use this dataset for non-commercial, academic purposes only: checkbox
  I agree not to distribute the dataset: checkbox

HESCAPE • DRVI models

HESCAPE (H&E + Spatial Contrastive Pretraining Benchmark) is a large-scale benchmark for multimodal learning in spatial transcriptomics.
This repository hosts the DRVI models for HESCAPE, organized by dataset panel.

---

Available DRVI Models

This dataset repo exposes the following drvi models and corresponding UMAP plots:

• human-5k-panel
• human-breast-panel
• human-colon-panel
• human-immuno-oncology-panel
• human-lung-healthy-panel
• human-multi-tissue-panel

Each model corresponds to an independent HESCAPE dataset gene panel.

---

Usage

The HESCAPE repository takes pretrained weights for pre-built images and genes to train the model. The directory structure is crucial for the training process to work correctly. The repository is structured as follows:

├── hescape (from github)
│   ├── README.md
│   ├── data
│   ├── experiments
│   ├── notebooks
│   ├── pyproject.toml
│   ├── src
│   ├── tests
│   ├── uv.lock
│   └── ...
├── pretrain_weights
│   ├── gene
│   │   ├── nicheformer
│   │   ├── drvi
│   │   └── <predefined gene models> ...
│   └── image
│        ├── h0-mini
│        ├── uni
│        └── <predefined image models> ...

Copy the drvi models as in in the pretrain_weights/gene/ folder as given in the directory. Further instructions on how to use HESCAPE are provided here

How to cite:

@misc{gindra2025largescalebenchmarkcrossmodallearning,
      title={A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics}, 
      author={Rushin H. Gindra and Giovanni Palla and Mathias Nguyen and Sophia J. Wagner and Manuel Tran and Fabian J Theis and Dieter Saur and Lorin Crawford and Tingying Peng},
      year={2025},
      eprint={2508.01490},
      archivePrefix={arXiv},
      primaryClass={q-bio.GN},
      url={https://arxiv.org/abs/2508.01490}, 
}

Contact:

• Rushin Gindra Helmholtz Munich, Munich (rushin.gindra@helmholtz-munich.de)
• The dataset is distributed under the Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0 Deed)