README.md

---
license: mit
library_name: diffusers
tags:
  - diffusion
  - inpainting
  - histopathology
  - medical-imaging
  - pathology
  - pytorch
pipeline_tag: image-to-image
---

# PathoGen - Histopathology Image Inpainting

PathoGen is a diffusion-based model for histopathology image inpainting. It enables realistic tissue pattern generation for filling masked regions in pathology whole slide images (WSI).

## Model Description

- **Model Type:** Diffusion model with custom attention processors
- **Task:** Image inpainting for histopathology images
- **Architecture:** UNet2DConditionModel with custom SkipAttnProcessor
- **Framework:** PyTorch, Diffusers, PyTorch Lightning

## Usage

### Installation

```bash
git clone https://github.com/mkoohim/PathoGen.git
cd PathoGen
pip install -r requirements.txt
```

### Download Weights

Download the attention weights and place them in your checkpoint directory:

```python
from huggingface_hub import hf_hub_download

# Download attention weights
hf_hub_download(
    repo_id="mkoohim/PathoGen",
    filename="attention.pt",
    local_dir="./checkpoints"
)
```

### Inference

```python
from src.models.pathogen import PathoGenModel
from omegaconf import OmegaConf
from PIL import Image

# Load configuration
config = OmegaConf.load("configs/config.yaml")

# Initialize model
model = PathoGenModel(config)
model.load_attention_weights("./checkpoints/attention.pt")
model.eval()

# Load images
image = Image.open("your_wsi_crop.jpg")
mask = Image.open("your_mask.jpg")
condition = Image.open("your_source_image.jpg")

# Run inference
result = model(image, mask, condition)
```

### Training

```bash
python train.py
```

See the [GitHub repository](https://github.com/mkoohim/PathoGen) for full training instructions.

## Model Files

| File | Description | Size |
|------|-------------|------|
| `attention.pt` | Trained attention module weights | ~190MB |

## Training Details

- **Base Model:** Stable Diffusion Inpainting UNet
- **Training Data:** Histopathology whole slide image crops
- **Optimizer:** AdamW
- **Learning Rate:** 1e-5
- **Precision:** Mixed precision (FP16)

## Intended Use

This model is designed for:
- Histopathology image inpainting and augmentation
- Research in computational pathology
- Data augmentation for pathology AI training


## Citation

```bibtex
@misc{pathogen2025,
  title={PathoGen: Diffusion-Based Synthesis of Realistic Lesions in Histopathology Images},
  author={mkoohim},
  year={2025},
  url={https://huggingface.co/mkoohim/PathoGen}
}
```

## License

This model is released under the MIT License.

## Links

- **GitHub:** [https://github.com/mkoohim/PathoGen](https://github.com/mkoohim/PathoGen)
- **Hugging Face:** [https://huggingface.co/mkoohim/PathoGen](https://huggingface.co/mkoohim/PathoGen)