ALHENA

The model weights of ALHENA, an H&E encoder distilled from transcriptomic foundation models for multi-scale slide analysis.

Package code on GitHub | Paper code on GitHub | Documentation | Prebuilt references

How to use

pip install alhena

import alhena

encoder = alhena.load_encoder()

On first use, load_encoder() downloads config.json, adapters.safetensors and heads.safetensors, checks them against the digests in config.json, and applies them to the UNI2-h model weights (see UNI2-h). The documentation covers tile embeddings, cell embeddings, cell-type annotation and multi-scale slide analysis.

Files

File	What it is
adapters.safetensors	The low-rank adapters, of rank 64: an update to the query and value projections of each of the 24 attention blocks of UNI2-h, 9.4 million parameters in all
heads.safetensors	The three projection heads used at inference, linear maps into the 512-dimensional cell space: image_proj for tile embeddings, img_tok_proj for cell embeddings and st_node_proj for single-cell reference embeddings
config.json	The architecture and image geometry, the identities of the encoder and of its cell space, and the SHA-256 digests of the two weight files and of the UNI2-h weights they apply to

UNI2-h

The UNI2-h model weights are not included. Their authors distribute them at MahmoodLab/UNI2-h, under CC BY-NC-ND 4.0 and their terms of use. Accept these terms there, download pytorch_model.bin, and set ALHENA_BACKBONE to the downloaded file. Weights already in the Hugging Face cache are found without it. See Set up the UNI2-h model weights.

Intended use

ALHENA is a research tool. It is not a medical device, and its outputs are not intended for clinical diagnosis or treatment decisions.

License

The ALHENA model weights, comprising the LoRA adapters and the projection heads used at inference, are released under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). The full text of the license is in LICENSE.

ALHENA requires the UNI2-h weights, which are not redistributed and must be obtained from MahmoodLab/UNI2-h under the license and terms of use of their authors.

The ALHENA software is released under the BSD 3-Clause License.

Citation

[TODO: add link to preprint]

Attribution under CC BY-NC 4.0 is given by citing the ALHENA preprint:

@article{alhena,
  title   = {{ALHENA}, an {H\&E} encoder distilled from transcriptomic foundation models
             for multi-scale slide analysis},
  author  = {Giraud-Sauveur, F{\'e}licie and Blampey, Quentin and Marza, Pierre and
             Benkirane, Hakim and Vakalopoulou, Maria and Courn{\`e}de, Paul-Henry and
             Christodoulidis, Stergios},
  journal = {TODO: preprint server},
  year    = {TODO},
  doi     = {TODO}
}

Please also cite UNI2-h, as its authors ask:

@article{chen2024uni,
  title   = {Towards a general-purpose foundation model for computational pathology},
  author  = {Chen, Richard J. and Ding, Tong and Lu, Ming Y. and Williamson, Drew F. K. and
             Jaume, Guillaume and Song, Andrew H. and Chen, Bowen and Zhang, Andrew and
             Shao, Daniel and Shaban, Muhammad and others},
  journal = {Nature Medicine},
  volume  = {30},
  number  = {3},
  pages   = {850--862},
  year    = {2024},
  doi     = {10.1038/s41591-024-02857-3}
}