--- license: cc-by-nc-nd-4.0 ---

Examples of tumor annotations in Merlin Plus

# Merlin-Cancer-Net Merlin-Cancer-Net is an nnU-Net trained with 423 tumor segmentation masks from the Merlin Plus dataset. It surpasses state-of-the-art public AI models and leading VLMs in the detection and segmentation of tumors in 9 organs: bladder, gallbladder, spleen, esophagus, stomach, duodenum, prostate, uterus, and adrenal glands. **Merlin Plus** is a dataset with longitudinal metadata (patient IDs and scan dates) and per-voxel annotations for organs and 9 tumor types in the Merlin dataset (Stanford, 25,494 CT scans). Merlin Plus is part of a collaboration between the Merlin Project at Stanford and the R-Super Project at Johns Hopkins University. It is available at HuggingFace: https://huggingface.co/datasets/AbdomenAtlas/MerlinPlus. Merlin-Cancer-Net was trained with 423 masks from the Merlin Plus training dataset. # Papers Scaling Artificial Intelligence for Multi-Tumor Early Detection with More Reports, Fewer Masks
Pedro R. A. S. Bassi, Xinze Zhou, Wenxuan Li, Szymon Płotka, Jieneng Chen, Qi Chen, Zheren Zhu, Jakub Prządo, Ibrahim E. Hamacı, Sezgin Er, Yuhan Wang, Ashwin Kumar, Bjoern Menze, Jarosław B. Ćwikła, Yuyin Zhou, Akshay S. Chaudhari, Curtis P. Langlotz, Sergio Decherchi, Andrea Cavalli, Kang Wang, Yang Yang, Alan L. Yuille, Zongwei Zhou*
*Johns Hopkins University, Stanford University, University of Zurich, University of Warsaw, Italian Institute of Technology, UCSF*
Preprint
**Merlin Plus: A Large-Scale, Multi-cancer, Image-Mask-Report Dataset** Pedro R. A. S. Bassi†, Wenxuan Li†, Szymon Płotka†, Ruby Honjol, Jakub Prządo, Xinze Zhou, Kang Wang, Yang Yang, Malte Jensen, Akshay S. Chaudhari, Curtis P. Langlotz, Alan L. Yuille, and Zongwei Zhou. *MICCAI 2026, LNCS 16895. Springer Nature Switzerland.* Paper PDF Merlin: A Vision Language Foundation Model for 3D Computed Tomography
Louis Blankemeier, Joseph Paul Cohen, Ashwin Kumar, Dave Van Veen, Syed Jamal Safdar Gardezi, Magdalini Paschali, Zhihong Chen, Jean-Benoit Delbrouck, Eduardo Reis, Cesar Truyts, Christian Bluethgen, Malte Engmann Kjeldskov Jensen, Sophie Ostmeier, Maya Varma, Jeya Maria Jose Valanarasu, Zhongnan Fang, Zepeng Huo, Zaid Nabulsi, Diego Ardila, Wei-Hung Weng, Edson Amaro Junior, Neera Ahuja, Jason Fries, Nigam H. Shah, Andrew Johnston, Robert D. Boutin, Andrew Wentland, Curtis P. Langlotz, Jason Hom, Sergios Gatidis, Akshay S. Chaudhari
*Stanford University*
*Nature, 2026.*
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Fabian Isensee, Paul F. Jaeger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
*German Cancer Research Center (DKFZ)*
*Nature Methods, 18(2), 203-211, 2021.*
# Instructions ### Installation ```bash conda create -n merlin_cancer_net python=3.12 -y conda activate merlin_cancer_net pip install torch torchvision torchaudio pip install nnunetv2 pandas tqdm nibabel itk ``` ### Data format Put every CT in one folder, as `.nii.gz` files. You can use any file name. ``` /path/to/dataset/ ├── case_1.nii.gz ├── case_2.nii.gz ... ``` ### Inference ```bash bash parallel_inference.sh \ --pth /path/to/dataset/ \ --outdir /path/to/joint/outputs/ \ --checkpoint Dataset307_radiologist_annotations_Merlin_Plus/nnUNetTrainer__nnUNetPlannerResEncL_torchres_isotropic__3d_fullres/ \ --gpus 0 ``` Pass several GPUs as `--gpus 0,1,2,3` to run in parallel. ### Split the joint labels The nnU-Net produces joint labels (one file for all segmentation classes). To split these labels: ```bash python split_labels_multi_tumor.py \ --input_dir /path/to/joint/outputs/ \ --output_dir /path/to/outputs/ ``` Outputs one folder per case, with one mask per tumor and organ: ``` /path/to/outputs/ ├── BDMAP_00000001/ | └── predictions/ | ├── spleen_lesion.nii.gz | ├── stomach_lesion.nii.gz | ├── liver.nii.gz | └── ... ``` # Citations ``` @article{bassi2025scaling, title={Scaling Artificial Intelligence for Multi-Tumor Early Detection with More Reports, Fewer Masks}, author={Bassi, Pedro R. A. S. and Zhou, Xinze and Li, Wenxuan and P{\l}otka, Szymon and Chen, Jieneng and Chen, Qi and Zhu, Zheren and Prz{\k{a}}do, Jakub and Hamamci, Ibrahim E. and Er, Sezgin and others}, journal={arXiv preprint arXiv:2510.14803}, year={2025} } @InProceedings{BasPed_Merlin_MICCAI2026, author = { Bassi, Pedro R. A. S. AND Li, Wenxuan AND Płotka, Szymon AND Honjol, Ruby AND Prządo, Jakub AND Zhou, Xinze AND Wang, Kang AND Yang, Yang AND Jensen, Malte AND Chaudhari, Akshay S. AND Langlotz, Curtis P. AND Yuille, Alan L. AND Zhou, Zongwei}, title = { { Merlin Plus: A Large-Scale, Multi-cancer, Image-Mask-Report Dataset } }, booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026}, year = {2026}, publisher = {Springer Nature Switzerland}, volume = {LNCS 16895}, month = {September}, page = {pending} } @article{blankemeier_kumar2026merlin, author = {Blankemeier, Louis and Kumar, Ashwin and Cohen, Joseph Paul and Liu, Jiaming and Liu, Longchao and Van Veen, Dave and Gardezi, Syed Jamal Safdar and Yu, Hongkun and Paschali, Magdalini and Chen, Zhihong and Delbrouck, Jean-Benoit and Reis, Eduardo and Holland, Robbie and Truyts, Cesar and Bluethgen, Christian and Wu, Yufu and Lian, Long and Jensen, Malte Engmann Kjeldskov and Ostmeier, Sophie and Varma, Maya and Valanarasu, Jeya Maria Jose and Fang, Zhongnan and Huo, Zepeng and Nabulsi, Zaid and Ardila, Diego and Weng, Wei-Hung and Amaro Junior, Edson and Ahuja, Neera and Fries, Jason and Shah, Nigam H. and Zaharchuk, Greg and Willis, Marc and Yala, Adam and Johnston, Andrew and Boutin, Robert D. and Wentland, Andrew and Langlotz, Curtis P. and Hom, Jason and Gatidis, Sergios and Chaudhari, Akshay S.}, title = {Merlin: a computed tomography vision-language foundation model and dataset}, journal = {Nature}, year = {2026}, doi = {10.1038/s41586-026-10181-8}, url = {https://doi.org/10.1038/s41586-026-10181-8} } @article{isensee2021nnunet, title={nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation}, author={Isensee, Fabian and Jaeger, Paul F. and Kohl, Simon A. A. and Petersen, Jens and Maier-Hein, Klaus H.}, journal={Nature Methods}, volume={18}, number={2}, pages={203--211}, year={2021} } ``` ## Acknowledgement This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research, the Patrick J. McGovern Foundation Award, and the National Institutes of Health (NIH) under Award Number R01EB037669. Paper content is covered by patents pending. © The Johns Hopkins University. This work is openly licensed via CC BY-NC-ND. Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License