Update README.md

34dff7c verified 3 days ago

4.16 kB

license: other
license_name: ohsu-non-commercial
license_link: https://github.com/ChangLab/miniMTI/blob/publication/LICENSE
tags:
  - biology
  - multiplex-imaging
  - virtual-staining
  - computational-pathology
  - cycif
extra_gated_prompt: >-
  You are about to access the miniMTI-CRC model weights. By clicking "Agree",
  you agree to the following terms: (1) You will use this model for
  non-commercial academic research purposes only. (2) You will not distribute,
  publish, or sublicense the model weights. (3) You will cite the miniMTI paper
  in any publications that use this model. Please use your institutional email
  address for your HuggingFace account.
extra_gated_fields:
  Name: text
  Affiliation: text
  I agree to use this model for non-commercial academic research only: checkbox

miniMTI-CRC: minimal multiplex tissue imaging for colorectal cancer

Pre-trained model weights for miniMTI trained on the publicly available RareCyte Orion CRC dataset (link). This model predicts missing immunofluorescence (IF) markers from a reduced antibody panel plus co-registered H&E.

Paper: bioRxiv 2026.01.21.700911 Code: GitHub Collection: miniMTI

Supported Markers

This model supports the following 17 IF markers + H&E (18 markers total):

Index	Marker	Index	Marker
0	DAPI	9	PD-L1
1	CD31	10	CD3e
2	CD45	11	CD163
3	CD68	12	E-cadherin
4	CD4	13	PD-1
5	FOXP3	14	Ki67
6	CD8a	15	PanCK
7	CD45RO	16	aSMA
8	CD20	17	H&E (RGB)

Any combination of these markers can be used as input, and the model will predict the remaining markers. The iterative panel selection algorithm identifies the most informative markers to measure experimentally.

Access

This model is released for non-commercial academic research use only. To access the model weights, you must:

Log in to HuggingFace
Agree to the license terms and share your contact information
Use your institutional email address

For programmatic access after approval:

from huggingface_hub import login
login()  # enter your HuggingFace token

Model Architecture

Component	Details
Backbone	RoBERTa (24 layers, 16 heads, dim=1024)
IF Tokenizer	VQGAN (codebook=256, latent=4x4)
H&E Tokenizer	VQGAN (codebook=256, latent=4x4)
Sequence length	18 markers x 16 tokens = 288 tokens
Training	Masked token prediction with cosine masking schedule
Training data	CRC-Orion (colorectal cancer WSIs, 17 IF + H&E)

Files

mvtm_model.ckpt — MVTM masked token model checkpoint (3.4 GB)
tokenizer/if_config.yaml — IF VQGAN configuration
tokenizer/if_model.ckpt — IF VQGAN checkpoint (955 MB)
tokenizer/he_config.yaml — H&E VQGAN configuration
tokenizer/he_model.ckpt — H&E VQGAN checkpoint (955 MB)
config.json — Model and tokenizer configuration

Usage

from eval.load_model import load_model_from_huggingface

model, tokenizer = load_model_from_huggingface(repo_id="changlab/miniMTI-CRC")

See the repository for full documentation.

Citation

@article{sims2026minimti,
  title={miniMTI: minimal multiplex tissue imaging enhances biomarker expression prediction from histology},
  author={Sims, Z. and Govindarajan, S. and Ait-Ahmad, K. and Ak, C. and Kuykendall, M. and Mills, G. B. and Eksi, E. and Chang, Y. H.},
  journal={bioRxiv},
  year={2026},
  doi={10.64898/2026.01.21.700911}
}

changlab
/

miniMTI-CRC