You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

You are about to access the miniMTI-CRC model weights. By clicking "Agree", you agree to the following terms: (1) You will use this model for non-commercial academic research purposes only. (2) You will not distribute, publish, or sublicense the model weights. (3) You will cite the miniMTI paper in any publications that use this model. Please use your institutional email address for your HuggingFace account.

miniMTI-CRC: minimal multiplex tissue imaging for colorectal cancer

Pre-trained model weights for miniMTI trained on the publicly available RareCyte Orion CRC dataset (link). This model predicts missing immunofluorescence (IF) markers from a reduced antibody panel plus co-registered H&E.

Paper: bioRxiv 2026.01.21.700911 Code: GitHub Collection: miniMTI

Supported Markers

This model supports the following 17 IF markers + H&E (18 markers total):

Index	Marker	Index	Marker
0	DAPI	9	PD-L1
1	CD31	10	CD3e
2	CD45	11	CD163
3	CD68	12	E-cadherin
4	CD4	13	PD-1
5	FOXP3	14	Ki67
6	CD8a	15	PanCK
7	CD45RO	16	aSMA
8	CD20	17	H&E (RGB)

Any combination of these markers can be used as input, and the model will predict the remaining markers. The iterative panel selection algorithm identifies the most informative markers to measure experimentally.

Access

This model is released for non-commercial academic research use only. To access the model weights, you must:

Log in to HuggingFace
Agree to the license terms and share your contact information
Use your institutional email address

For programmatic access after approval:

from huggingface_hub import login
login()  # enter your HuggingFace token

Model Architecture

Component	Details
Backbone	RoBERTa (24 layers, 16 heads, dim=1024)
IF Tokenizer	VQGAN (codebook=256, latent=4x4)
H&E Tokenizer	VQGAN (codebook=256, latent=4x4)
Sequence length	18 markers x 16 tokens = 288 tokens
Training	Masked token prediction with cosine masking schedule
Training data	CRC-Orion (colorectal cancer WSIs, 17 IF + H&E)

Files

mvtm_model.ckpt — MVTM masked token model checkpoint (3.4 GB)
tokenizer/if_config.yaml — IF VQGAN configuration
tokenizer/if_model.ckpt — IF VQGAN checkpoint (955 MB)
tokenizer/he_config.yaml — H&E VQGAN configuration
tokenizer/he_model.ckpt — H&E VQGAN checkpoint (955 MB)
config.json — Model and tokenizer configuration

Usage

from eval.load_model import load_model_from_huggingface

model, tokenizer = load_model_from_huggingface(repo_id="changlab/miniMTI-CRC")

See the repository for full documentation.

Citation

@article{sims2026minimti,
  title={miniMTI: minimal multiplex tissue imaging enhances biomarker expression prediction from histology},
  author={Sims, Z. and Govindarajan, S. and Ait-Ahmad, K. and Ak, C. and Kuykendall, M. and Mills, G. B. and Eksi, E. and Chang, Y. H.},
  journal={bioRxiv},
  year={2026},
  doi={10.64898/2026.01.21.700911}
}

License

Downloads last month: 6

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including changlab/miniMTI-CRC

miniMTI

Collection

Minimal multiplex tissue imaging: virtual staining from H&E plus a minimal set of measured markers. • 2 items • Updated 3 days ago