Add HuggingFace model card for HeteroTissueDiffuse

Includes YAML frontmatter (license, tags, pipeline), full model description,
8-channel conditioning explanation, performance tables, training details,
and quick-start inference instructions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.gitattributes

@@ -0,0 +1,9 @@
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.gif filter=lfs diff=lfs merge=lfs -text
+*.mp4 filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,214 @@
+---
+license: apache-2.0
+language:
+  - en
+tags:
+  - histopathology
+  - diffusion
+  - image-generation
+  - medical-imaging
+  - latent-diffusion
+  - semantic-synthesis
+  - tissue-synthesis
+  - computational-pathology
+  - stable-diffusion
+datasets:
+  - Camelyon16
+  - PANDA
+  - TCGA
+pipeline_tag: image-to-image
+library_name: diffusers
+model_name: HeteroTissueDiffuse
+arxiv: 2509.17847
+---
+
+# HeteroTissueDiffuse
+
+**Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology**
+
+*NeurIPS 2025*
+
+[Saghir Alfasly](https://saghiralfasly.github.io/) · [Wataru Uegami](https://www.linkedin.com/in/wataru-uegami-8b106920a/) · [MD Enamul Hoq](https://www.linkedin.com/in/mhoq89/) · [Ghazal Alabtah](https://www.linkedin.com/in/ghazal-alabtah-00/) · [H.R. Tizhoosh](https://tizhoosh.com/)
+
+KIMIA Lab, Mayo Clinic, Rochester, MN, USA
+
+[![Paper](https://img.shields.io/badge/arXiv-2509.17847-b31b1b.svg)](https://arxiv.org/abs/2509.17847)
+[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://kimialabmayo.github.io/hetero_tissue_diffuse_page/)
+[![GitHub](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/KimiaLabMayo/HeteroTissueDiffuse)
+
+---
+
+![demo](https://raw.githubusercontent.com/KimiaLabMayo/HeteroTissueDiffuse/main/assets/illustrations/demo_gif.gif)
+
+---
+
+## Model Description
+
+**HeteroTissueDiffuse** is a latent diffusion model (LDM) that synthesizes heterogeneous histopathology images by conditioning on both a **binary semantic map** and **raw tissue crop exemplars**. Unlike text- or embedding-guided approaches, it injects actual tissue appearance directly into the diffusion process, preserving staining characteristics, nuclear morphology, and cellular texture.
+
+The model addresses a fundamental limitation of prior generative methods in histopathology: the tendency to produce homogeneous (single-tissue-type) images. By providing spatially-registered visual prompts for each tissue class, the model generates realistic heterogeneous slides that accurately reflect real-world tissue organization.
+
+### Architecture
+
+- **Base**: CompVis Latent Diffusion Model with VQ-regularized autoencoder
+- **First stage**: `VQModelInterface` (3-channel latent, 8192 codebook)
+- **Conditioning encoder**: `SpatialRescaler` with `in_channels=8` (replaces ADE20K default of 182)
+- **U-Net**: 128 base channels, attention at resolutions 32/16/8
+- **Image size**: 256×256 pixels
+- **Sampling**: DDIM, 200 steps, η=1
+
+### 8-Channel Conditioning Tensor
+
+```
+Channel 0:    normal onehot mask         (1 where segmentation == 0)
+Channels 1–3: normal tissue crop RGB     (float32, normalized to [-1,1])
+Channel 4:    tumor onehot mask          (1 where segmentation == 1)
+Channels 5–7: tumor tissue crop RGB      (float32, normalized to [-1,1])
+```
+
+The tissue crops are small patches (typically 30–60px) extracted from a reference slide and pasted spatially within the corresponding mask region. This lets users control staining appearance at inference time without any fine-tuning.
+
+---
+
+## Available Checkpoints
+
+| File | Dataset | Description |
+|------|---------|-------------|
+| `camelyon16/epoch=000064.ckpt` | Camelyon16 | Binary tumor/normal masks, 256×256 |
+| `panda/epoch=XXXXXX.ckpt` | PANDA | Gleason grading regions *(coming soon)* |
+| `tcga/epoch=XXXXXX.ckpt` | TCGA (self-supervised) | 100 pseudo-phenotype clusters *(coming soon)* |
+
+---
+
+## Quick Start
+
+### 1. Clone the inference code
+
+```bash
+git clone https://github.com/CompVis/stable-diffusion.git
+cd stable-diffusion
+
+# Apply the 2 required patches (see GitHub README for details)
+# Then copy our inference script
+wget https://raw.githubusercontent.com/KimiaLabMayo/HeteroTissueDiffuse/main/inference_heteroTissueDiffuse_camelyon.py
+```
+
+### 2. Download the checkpoint
+
+```bash
+pip install huggingface_hub
+
+python - <<'EOF'
+from huggingface_hub import hf_hub_download
+hf_hub_download(
+    repo_id="KimiaLabMayo/HeteroTissueDiffuse",
+    filename="camelyon16/epoch=000064.ckpt",
+    local_dir="inference/"
+)
+EOF
+```
+
+Or via CLI:
+```bash
+huggingface-cli download KimiaLabMayo/HeteroTissueDiffuse \
+    camelyon16/epoch=000064.ckpt --local-dir inference/
+```
+
+### 3. Run inference
+
+```bash
+conda activate diff   # PyTorch 2.0.1 + CUDA 11+
+
+python inference_heteroTissueDiffuse_camelyon.py \
+    --normal_prompt  inference/promptNormal2.png \
+    --tumor_prompt   inference/promptTumor2.png \
+    --segmentation_root inference/masks \
+    --ckpt_path      inference/epoch=000064.ckpt \
+    --output_dir     outputs/inference_results
+```
+
+**Inputs:**
+- `--normal_prompt` / `--tumor_prompt`: small PNG crops of representative tissue regions (provided as examples in the [GitHub repo](https://github.com/KimiaLabMayo/HeteroTissueDiffuse/tree/main/inference))
+- `--segmentation_root`: folder of `.npy` binary masks (512×512, dtype bool, 0=normal, 1=tumor)
+
+**Outputs** (per mask):
+- `frame_XXX.png` — generated histopathology image
+- `prompt_frame_XXX.png` — visualization of the conditioning (mask + overlaid crops)
+
+---
+
+## Performance
+
+### Fréchet Distance (lower is better)
+
+| Method | Camelyon16 | PANDA | TCGA |
+|--------|-----------|-------|------|
+| Unconditional LDM | 430.1 | — | — |
+| **HeteroTissueDiffuse (ours)** | **72.0** | ↓ 2–3× | ↓ 2–3× |
+
+### Downstream Segmentation (IoU)
+
+| Training data | Camelyon16 | PANDA |
+|---------------|-----------|-------|
+| Real images | 0.72 | 0.96 |
+| **Synthetic (ours)** | **0.71** | **0.95** |
+| Synthetic (no conditioning) | 0.51 | 0.82 |
+
+### Pathologist Assessment
+
+A certified pathologist evaluated 120 images in a blinded study. Synthetic images conditioned with visual prompts received quality scores **indistinguishable from real images**:
+
+> *"The generated images tended to have equal or higher quality than the real images."*
+
+---
+
+## Intended Use
+
+- **Research**: generating large annotated synthetic histopathology datasets for downstream model training
+- **Augmentation**: expanding small annotated datasets with realistic diverse tissue variations
+- **Privacy-preserving data sharing**: synthetic data as a substitute for patient slides
+- **Education**: illustrating tissue morphology variations
+
+### Out-of-scope use
+
+- Clinical diagnosis or patient care — this is a research model
+- Generating images to deceive or misrepresent clinical findings
+
+---
+
+## Training Details
+
+### Camelyon16 Checkpoint
+
+- **Dataset**: Camelyon16 (lymph node whole-slide images, binary tumor/normal segmentation)
+- **Patch size**: 256×256 pixels at 0.5 µm/px
+- **Training steps**: 64 epochs
+- **Optimizer**: Adam, lr=1e-6
+- **Hardware**: A100 GPU
+- **Framework**: PyTorch 2.0.1 + pytorch-lightning 1.4.2
+
+### Self-Supervised Extension (TCGA)
+
+Patches from 11,765 TCGA whole-slide images were embedded using a histopathology foundation model (PathDino), then clustered into 100 tissue phenotypes via k-means. These clusters form pseudo-semantic maps for training without manual annotation.
+
+---
+
+## Citation
+
+```bibtex
+@InProceedings{Alfasly2025HeteroTissueDiffuse,
+    author    = {Alfasly, Saghir and Uegami, Wataru and Hoq, MD Enamul and Alabtah, Ghazal and Tizhoosh, H.R.},
+    title     = {Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology},
+    booktitle = {Neural Information Processing Systems (NeurIPS)},
+    month     = {December},
+    year      = {2025}
+}
+```
+
+---
+
+## License
+
+Apache 2.0. Model weights are released for non-commercial research use. See [LICENSE](LICENSE) for details.
+
+The base architecture follows [CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion) (MIT License).