README.md

---
license: other
license_name: ohsu-non-commercial
license_link: https://github.com/ChangLab/miniMTI/blob/publication/LICENSE
tags:
  - biology
  - multiplex-imaging
  - virtual-staining
  - computational-pathology
  - cycif
extra_gated_prompt: >-
  You are about to access the miniMTI-CRC model weights. By clicking "Agree",
  you agree to the following terms: (1) You will use this model for
  non-commercial academic research purposes only. (2) You will not distribute,
  publish, or sublicense the model weights. (3) You will cite the miniMTI paper
  in any publications that use this model. Please use your institutional email
  address for your HuggingFace account.
extra_gated_fields:
  Name: text
  Affiliation: text
  I agree to use this model for non-commercial academic research only: checkbox
---

# miniMTI-CRC: minimal multiplex tissue imaging for colorectal cancer

Pre-trained model weights for **miniMTI** trained on the publicly available **RareCyte Orion CRC dataset** [(link)](https://www.tissue-atlas.org/atlas-datasets/lin-chen-campton-2023/#data-access). This model predicts missing immunofluorescence (IF) markers
from a reduced antibody panel plus co-registered H&E.

**Paper:** [bioRxiv 2026.01.21.700911](https://www.biorxiv.org/content/10.64898/2026.01.21.700911v1)
**Code:** [GitHub](https://github.com/ChangLab/miniMTI)
**Collection:** [miniMTI](https://huggingface.co/collections/changlab/minimti-69b37b060f38b7c593eea196)

## Supported Markers

This model supports the following 17 IF markers + H&E (18 markers total):

| Index | Marker | Index | Marker |
|-------|--------|-------|--------|
| 0 | DAPI | 9 | PD-L1 |
| 1 | CD31 | 10 | CD3e |
| 2 | CD45 | 11 | CD163 |
| 3 | CD68 | 12 | E-cadherin |
| 4 | CD4 | 13 | PD-1 |
| 5 | FOXP3 | 14 | Ki67 |
| 6 | CD8a | 15 | PanCK |
| 7 | CD45RO | 16 | aSMA |
| 8 | CD20 | 17 | H&E (RGB) |

Any combination of these markers can be used as input, and the model will predict
the remaining markers. The iterative panel selection algorithm identifies the most
informative markers to measure experimentally.

## Access

This model is released for **non-commercial academic research use only**.
To access the model weights, you must:

1. Log in to HuggingFace
2. Agree to the license terms and share your contact information
3. Use your institutional email address

For programmatic access after approval:

```python
from huggingface_hub import login
login()  # enter your HuggingFace token
```

## Model Architecture

| Component | Details |
|-----------|---------|
| Backbone  | RoBERTa (24 layers, 16 heads, dim=1024) |
| IF Tokenizer | VQGAN (codebook=256, latent=4x4) |
| H&E Tokenizer | VQGAN (codebook=256, latent=4x4) |
| Sequence length | 18 markers x 16 tokens = 288 tokens |
| Training | Masked token prediction with cosine masking schedule |
| Training data | CRC-Orion (colorectal cancer WSIs, 17 IF + H&E) |

## Files

- `mvtm_model.ckpt` — MVTM masked token model checkpoint (3.4 GB)
- `tokenizer/if_config.yaml` — IF VQGAN configuration
- `tokenizer/if_model.ckpt` — IF VQGAN checkpoint (955 MB)
- `tokenizer/he_config.yaml` — H&E VQGAN configuration
- `tokenizer/he_model.ckpt` — H&E VQGAN checkpoint (955 MB)
- `config.json` — Model and tokenizer configuration

## Usage

```python
from eval.load_model import load_model_from_huggingface

model, tokenizer = load_model_from_huggingface(repo_id="changlab/miniMTI-CRC")
```

See the [repository](https://github.com/ChangLab/miniMTI)
for full documentation.

## Citation

```bibtex
@article{sims2026minimti,
  title={miniMTI: minimal multiplex tissue imaging enhances biomarker expression prediction from histology},
  author={Sims, Z. and Govindarajan, S. and Ait-Ahmad, K. and Ak, C. and Kuykendall, M. and Mills, G. B. and Eksi, E. and Chang, Y. H.},
  journal={bioRxiv},
  year={2026},
  doi={10.64898/2026.01.21.700911}
}
```

## License

Copyright (c) 2024 – Present, Oregon Health & Science University (OHSU). All rights reserved.
This model is licensed for non-commercial academic research use only.
See the [full license](https://github.com/ChangLab/miniMTI/blob/publication/LICENSE) for details.