---
license: apache-2.0
library_name: diffusers
pipeline_tag: text-to-image
tags:
  - remote-sensing
  - diffusion
  - controlnet
  - custom-pipeline
language:
  - en
---

> [!WARNING] we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via bili_sakura@zju.edu.cn

# BiliSakura/CRS-Diff

Diffusers-style packaging for the CRS-Diff checkpoint, with a custom Hugging Face `DiffusionPipeline` implementation.

## Model Details

- **Base project**: `CRS-Diff` (Controllable Remote Sensing Image Generation with Diffusion Model)
- **Checkpoint source**: `/root/worksapce/models/raw/CRS-Diff/last.ckpt`
- **Pipeline class**: `CRSDiffPipeline` (in `pipeline.py`)
- **Scheduler**: `DDIMScheduler`
- **Resolution**: 512x512 (default in training/inference config)

## Repository Structure

```text
CRS-Diff/
  pipeline.py
  modular_pipeline.py
  crs_core/
    autoencoder.py
    text_encoder.py
    local_adapter.py
    global_adapter.py
    metadata_embedding.py
    modules/
  model_index.json
  scheduler/
    scheduler_config.json
  unet/
  vae/
  text_encoder/
  local_adapter/
  global_content_adapter/
  global_text_adapter/
  metadata_encoder/
```

## Usage

Install dependencies first:

```bash
pip install diffusers transformers torch torchvision omegaconf einops safetensors pytorch-lightning
```

Load the pipeline (local path or Hub repo), then run inference:

```python
import torch
import numpy as np
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "/root/worksapce/models/BiliSakura/CRS-Diff",
    custom_pipeline="pipeline.py",
    trust_remote_code=True,
    model_path="/root/worksapce/models/BiliSakura/CRS-Diff",
)
pipe = pipe.to("cuda")

# Example placeholder controls; replace with real CRS controls.
b = 1
local_control = torch.zeros((b, 18, 512, 512), device="cuda", dtype=torch.float32)
global_control = torch.zeros((b, 1536), device="cuda", dtype=torch.float32)
metadata = torch.zeros((b, 7), device="cuda", dtype=torch.float32)

out = pipe(
    prompt=["a remote sensing image of an urban area"],
    negative_prompt=["blurry, distorted, overexposed"],
    local_control=local_control,
    global_control=global_control,
    metadata=metadata,
    num_inference_steps=50,
    guidance_scale=7.5,
    eta=0.0,
    strength=1.0,
    global_strength=1.0,
    output_type="pil",
)
image = out.images[0]
image.save("crs_diff_sample.png")
```

## Notes

- This repository is packaged in a diffusers-compatible layout with a custom pipeline.
- Loading path follows the same placeholder-aware custom pipeline pattern as HSIGene.
- Split component weights are provided in diffusers-style folders (`unet/`, `vae/`, adapters, and encoders).
- Monolithic `crs_model/last.ckpt` fallback is intentionally removed; this repo is split-components only.
- Legacy external source trees (`models/`, `ldm/`) are removed; runtime code is in lightweight `crs_core/`.
- `CRSDiffPipeline` expects CRS-specific condition tensors (`local_control`, `global_control`, `metadata`).
- If you publish to Hugging Face Hub, keep `trust_remote_code=True` when loading.

## Citation

```bibtex
@article{tang2024crs,
  title={Crs-diff: Controllable remote sensing image generation with diffusion model},
  author={Tang, Datao and Cao, Xiangyong and Hou, Xingsong and Jiang, Zhongyuan and Liu, Junmin and Meng, Deyu},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  year={2024},
  publisher={IEEE}
}
```