README.md

---
license: apache-2.0
pipeline_tag: image-segmentation
tags:
- cell-classification
- cell-detection
- digital-pathology
- whole-slide-image
- H&E
- computational-pathology
- pytorch
---
                  

# HNE2Cell — H&E Whole-Slide Image Cell Detection & Classification

HNE2Cell detects and classifies **16 cell types** from H&E-stained whole-slide images (WSI).
It takes 256×256 px patches as input and outputs per-cell contours, centroids, and type labels.

## Cell Types (16 classes)

| ID | Cell Type | Color |
|----|-----------|-------|
| 0 | Background | — |
| 1 | **Malignant** | 🔴 Red |
| 2 | CD4 T | 🔵 Dodger Blue |
| 3 | CD8 T | 🔵 Royal Blue |
| 4 | B | 🔵 Blue |
| 5 | Plasma | 🔵 Cornflower Blue |
| 6 | Macrophage | 🔵 Powder Blue |
| 7 | Myeloid | 🔵 Steel Blue |
| 8 | DC | 🔵 Deep Sky Blue |
| 9 | Fibroblast | 🟢 Forest Green |
| 10 | Endothelial | 🟢 Medium Sea Green |
| 11 | Pericyte | 🟢 Lime Green |
| 12 | Epithelial | 🟠 Dark Orange |
| 13 | Immune_Other | ⚪ Light Blue |
| 14 | Stromal_Other | 🟤 Olive Drab |
| 15 | Dead | ⚫ Grey |

## Pipeline Overview

The full pipeline consists of three steps:

```
┌───────────────┐       ┌────────────────┐      ┌──────────────────┐
│ 1. Normalize │ ──→   │  2. Patchify  │ ──→  │  3. Inference   │
│   (Reinhard) │       │ (256px, 64ov) │     │ (Cell Detection)│
└───────────────┘       └────────────────┘      └──────────────────┘
    SVS / TIF             PNG patches          Masks + Centroids
```

### Magnification

| | 40x (Recommended) | 20x (Supported) |
|---|---|---|
| **Accuracy** | Best — fine-grained cell boundaries | Good — may miss small immune cells |
| **Speed** | More patches per slide | Fewer patches, faster |
| **Use when** | Immune cell subtyping matters | Quick screening / large cohorts |

> **40x is strongly recommended.** The model was primarily trained on 40x data. 20x works but expect reduced precision for small cells (lymphocytes, DCs).

---

## System Requirements

### Software dependencies (tested versions)

Core packages (as reported in the manuscript):

- Python 3.10
- pytorch == 2.5.1
- timm == 1.0.8
- transformers == 4.44.0
- scanpy == 1.10.3
- squidpy == 1.5.0
- spatialdata == 0.2.5
- scikit-image == 0.24.0
- scikit-learn == 1.2.2
- scipy == 1.13.1
- shapely == 2.0.7

Additional utilities required by the pipeline scripts:

- torchvision (matching the PyTorch 2.5.1 release)
- tifffile, Pillow, opencv-python-headless, pandas, tqdm
- huggingface_hub
- openslide-python (optional, for `.svs` files)

### Operating systems tested

- Ubuntu 22.04 LTS
- Ubuntu 20.04 LTS

(Not tested on Windows/macOS.)

### Hardware requirements

> **Note:** WSI processing is memory-intensive. This pipeline is designed for
> server- or workstation-class hardware, not standard desktops.

**Minimum (small WSIs, ~1–2 GB):**
- GPU: NVIDIA GPU with ≥12 GB VRAM
- RAM: 32 GB (64 GB strongly recommended)
- Disk: 100 GB free

**Recommended (typical WSIs, 2–10 GB):**
- GPU: NVIDIA A100 / RTX 4090 / RTX 3090 (≥24 GB VRAM)
- RAM: ≥128 GB
- Disk: 500 GB+ free (intermediate `Aligned-hne.tif` can be 20–50 GB per slide)

**Tested configurations:**
- NVIDIA A100 (40 GB VRAM), 256 GB RAM, Ubuntu 22.04
- NVIDIA RTX 3060 (12 GB VRAM), 64 GB RAM, Ubuntu 22.04

CPU-only inference is not supported in practice — full WSI inference would take
days even on a high-core-count CPU.

---

## Installation Guide

### Recommended: Conda environment from `cellvit_rv3.yml`

The repository includes a frozen conda environment file with all dependencies pinned
to the exact versions used in the manuscript.

```bash
# 1. Download environment file
wget https://huggingface.co/roobee79/HNE2Cell/resolve/main/cellvit_rv3.yml

# 2. Create environment
conda env create -f cellvit_rv3.yml

# 3. Activate
conda activate cellvit_rv3
```

**Typical install time:** ~10–15 minutes on a Linux server with a stable network connection
(dominated by the PyTorch + CUDA toolkit download).


### Download the model

```python
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="roobee79/HNE2Cell",
    filename="HNE2cell_pub_patch73_jit.pt"
)
```

---

## Demo: Reproducible Walkthrough

To verify your installation, run the pipeline on the example slide included in this
repository (`TCGA-56-8628-01Z-00-DX1`, LUSC, ~36 MB).

### Download the model, example slide, and reference image

```python
from huggingface_hub import hf_hub_download

REPO_ID = "roobee79/HNE2Cell"

model_path = hf_hub_download(
    repo_id=REPO_ID,
    filename="HNE2cell_pub_patch73_jit.pt"
)

slide_path = hf_hub_download(
    repo_id=REPO_ID,
    filename="TCGA-56-8628-01Z-00-DX1.AAC57164-E0F9-4DF0-87EA-5C50FB201895.svs"
)

ref_path = hf_hub_download(
    repo_id=REPO_ID,
    filename="standard-ilc.tif"
)
```

### Run the pipeline

```bash
# Place the downloaded slide in a working directory
mkdir -p example/slides
cp <slide_path> example/slides/

# Step 1: Normalize
python normalize.py \
    --input_dir ./example/slides \
    --target ./standard-ilc.tif

# Step 2: Patchify at 40x
python patchify.py \
    --input_dir ./example/slides \
    --magnification 40 \
    --patch_size 256 \
    --overlap 64 \
    --workers 8

# Step 3: Inference
python inference.py \
    --input_dir ./example/slides/TCGA-56-8628-01Z-00-DX1.../patches \
    --output_dir ./example/results \
    --model_path ./HNE2cell_pub_patch73_jit.pt \
    --magnification 40 \
    --batch_size 32
```

### Expected output

```
example/results/
├── Aligned-hne.tif        # Normalized full-resolution H&E
├── Aligned-hne.jpg        # 4× preview
├── patch_*_mask.png       # Per-patch cell type masks
└── patch_*_centroid.csv   # Cell centroids with type labels
```

**Expected results on the example slide (`TCGA-56-8628-01Z-00-DX1`):**
Approximately **63,000 cells** are detected across the 16 classes.
Small variation (±a few percent) is expected between hardware configurations.

### Expected runtime

| Hardware | Full pipeline runtime |
|---|---|
| NVIDIA A100 (40 GB) + 256 GB RAM | ~20 min |
| NVIDIA RTX 3060 (12 GB) + 64 GB RAM | ~30 min |

A system without sufficient RAM (<32 GB) will fail at the normalization step
due to full-resolution image loading.

> The example slide is from **TCGA-LUSC** and is redistributed under the
> [NIH Genomic Data Sharing Policy](https://sharing.nih.gov/genomic-data-sharing-policy).

---

## Instructions for Use (On Your Own Data)

```bash
# Step 1: Color normalization (Reinhard method)
python normalize.py \
    --input_dir /path/to/slides \
    --target /path/to/standard-ilc.tif

# Step 2: Extract patches (40x recommended)
python patchify.py \
    --input_dir /path/to/slides \
    --magnification 40 \
    --patch_size 256 \
    --overlap 64 \
    --workers 8

# Step 3: Cell detection & classification
python inference.py \
    --input_dir /path/to/patch_folders \
    --output_dir /path/to/results \
    --model_path ./HNE2cell_all_patch73_jit.pt \
    --magnification 40 \
    --batch_size 32
```

---

## Input / Output Details

### Input

| Step | Input | Format |
|------|-------|--------|
| Normalize | Raw WSI | `.svs`, `.tif`, `.tiff`, `.ndpi` |
| Patchify | Normalized image | `Aligned-hne.tif` (from Step 1) |
| Inference | Patches | `256×256 px` PNG files |

### Output

| File | Description |
|------|-------------|
| `Aligned-hne.tif` | Full-resolution normalized H&E image |
| `Aligned-hne.jpg` | 4× downsampled preview |
| `recon.tif` | Tissue-only reconstruction (intermediate) |
| `*_mask.png` | Per-patch cell segmentation mask (colored by type) |
| `*_centroid.csv` | Cell centroids with columns: `slide_id, x, y, celltype, celltype_name` |

### Centroid CSV format

```csv
slide_id,x,y,celltype,celltype_name
patch_0_0,112.3,87.5,1,Malignant
patch_0_0,45.1,201.2,2,CD4T
...
```

To convert patch-local coordinates to WSI-global coordinates:

```python
# Parse patch filename: {prefix}_{x_offset}_{y_offset}.png
x_global = x + x_offset
y_global = y + y_offset
```

---

## Normalization Reference Image

The pipeline uses Reinhard color normalization in LAB color space. You need a reference image
(`standard-ilc.tif`) that represents your target stain appearance. The reference image is
included in this repository, or you can supply your own.

---

## File Structure

```
HNE2Cell/
├── README.md                                                    # This file
├── HNE2cell_pub_patch73_jit.pt                                  # TorchScript model
├── normalize.py                                                 # Step 1: Reinhard normalization
├── patchify.py                                                  # Step 2: Patch extraction
├── inference.py                                                 # Step 3: Model inference
├── post_processing.py                                           # Cell post-processing module
├── tools.py                                                     # Utility functions
├── standard-ilc.tif                                             # Reference image for normalization
└── TCGA-56-8628-01Z-00-DX1.AAC57164-E0F9-4DF0-87EA-5C50FB201895.svs   # Example slide (TCGA-LUSC)
```

---

  
## Citation

If you use HNE2Cell in your research, please cite:

```bibtex
@misc{hne2cell,
  title={Spatial transcriptomics–supervised deep learning enables single-cell mapping of tumor immune architecture from routine histology},
  year={2026},
  url={https://huggingface.co/roobee79/HNE2Cell}
}
```

The example slide is derived from data generated by the TCGA:
<https://portal.gdc.cancer.gov/>.


## License

This repository uses a dual licensing scheme commonly adopted in academic ML/biomedical projects (e.g., SAM, LLaMA):

- **Source code** (`.py` files): Released under the **MIT License**. See `LICENSE`.
- **Model weights** (`HNE2cell_pub_patch73_jit.pt`): Released under 
  **Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)**. 
  See `MODEL_LICENSE`. Free for academic and non-commercial research use.
  
  For commercial licensing, please contact:  
  **Ewha University-Industry Collaboration Foundation**  
  Technology Commercialization Team  
  Ewha Womans University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul, Republic of Korea  
  Web: <https://research.ewha.ac.kr> | <https://epm.ewha.ac.kr>
- **Example slide** (`TCGA-56-8628-*.svs`): Derived from TCGA-LUSC, governed by 
  the [NIH Genomic Data Sharing Policy](https://sharing.nih.gov/genomic-data-sharing-policy).