Model Card for TMEsegBreastTCGA

What is TMEsegBreastTCGA?

TMEsegBreastTCGA is a TensorFlow SegFormer semantic segmentation model for breast cancer histology images. The model was fine-tuned to segment tissue compartments in H&E-stained breast cancer image regions using TCGA breast cancer data from the Breast Cancer Semantic Segmentation (BCSS) resource.

This repository contains the TensorFlow checkpoint files needed to load the model with the Hugging Face transformers library.

Model Description

• Model type: SegFormer semantic segmentation model
• Backbone/checkpoint family: MiT-B3 / SegFormer
• Framework: TensorFlow
• Task: Multi-class semantic segmentation of breast cancer histology image regions
• Input: H&E breast cancer histology image patches/regions
• Output: Per-pixel class logits for tissue semantic classes
• Training data: TCGA breast cancer histology data distributed through the BCSS dataset
• License: MIT

The exact class mapping is stored in config.json and can be inspected with:

from transformers import AutoConfig

repo_id = "idso-fa1-pathology/TMEsegBreastTCGA"
config = AutoConfig.from_pretrained(repo_id, subfolder="checkpoint8class") # subfolder="checkpoint9class"

print(config.num_labels)
print(config.id2label)

Files

README.md                         # model card

checkpoint9class/
├── config.json                    # model configuration, including label mapping
└── tf_model.h5                    # TensorFlow model weights

checkpoint8class/
├── config.json                    # model configuration, including label mapping
└── tf_model.h5                    # TensorFlow model weights

Intended Use

Download the checkpoint to the local for downsream finetuning or inference.

Install the Hugging Face Hub client if needed:

pip install -U huggingface_hub

from pathlib import Path
import shutil
from huggingface_hub import hf_hub_download

repo_id = "idso-fa1-pathology/TMEsegBreastTCGA"
subfolder = "checkpoint9class"
output_dir = Path("./tmeseg-infer/model/mit-b3-finetuned-TCGAbcss9class")
output_dir.mkdir(parents=True, exist_ok=True)

for filename in ["config.json", "tf_model.h5"]:
    downloaded_file = hf_hub_download(
        repo_id=repo_id,
        filename=filename,
        subfolder=subfolder,
        repo_type="model",
    )
    shutil.copy(downloaded_file, output_dir / filename)

print(f"Checkpoint saved to: {output_dir}")

This model is intended for research use in computational pathology, including:

• semantic segmentation of breast cancer H&E image regions;
• tissue compartment mapping for tumor microenvironment analysis;
• downstream quantitative pathology workflows using tissue-level segmentation masks.

Out-of-Scope Use

This model is not intended for:

• standalone clinical diagnosis;
• treatment selection without independent clinical validation;
• deployment on clinical slides without site-specific quality control and validation;
• use on non-breast tissues or non-H&E images without further validation;
• use as a substitute for review by trained pathology professionals.

Training Data

The model was trained using TCGA breast cancer histology data from the Breast Cancer Semantic Segmentation (BCSS) dataset. The BCSS resource is associated with the paper:

Amgad M, Elfandy H, et al. Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics. 2019. doi: 10.1093/bioinformatics/btz083.

The BCSS GitHub repository provides dataset download instructions, metadata, masks, RGB image extraction utilities, and label-code information:

https://github.com/PathologyDataScience/BCSS

Limitations

• The model was trained on TCGA/BCSS breast cancer histology data and may not generalize to other institutions, scanners, staining protocols, tissue processing protocols, magnifications, or image resolutions.
• Performance may degrade on out-of-distribution samples, artifacts, poor-quality tissue regions, rare morphologies, or tissue classes underrepresented in the training data.
• The model outputs semantic tissue classes, not cell-level annotations.
• The label ontology should be verified through config.id2label before downstream use.
• The model should be externally validated before use in new cohorts or clinical/translational studies.