---
license: apache-2.0
library_name: rsdiff
tags:
  - diffusion
  - text-to-image
  - remote-sensing
  - satellite-imagery
  - cascaded-diffusion
  - rsicd
datasets:
  - arampacha/rsicd
language:
  - en
pipeline_tag: text-to-image
inference: false
---

# rsdiff-sr-cascade-ep650

A T5-conditioned cascaded diffusion model for **text-to-satellite-image
generation** at 256×256, trained on RSICD.

- **FID 65.70** on the full RSICD test split (N=1093, Inception
  feature=2048, cascade-256, `cond_scale=5.0`).
- **CLIP-score 0.278** (OpenAI ViT-B/32), with a +0.046 lift over a
  shuffled-caption null baseline.

Code & full tech report: https://github.com/asebaq/rsdiff
([REPORT.md](https://github.com/asebaq/rsdiff/blob/main/docs/REPORT.md)).

## Architecture

Two-stage Imagen-style cascade conditioned on a frozen T5-base text
encoder.

| Stage | Params | Resolution | Conditioning |
| --- | --- | --- | --- |
| LR base UNet | 27.18 M | 128×128 | T5-base, `p_uncond=0.1` |
| SR UNet | 92.66 M | 128 → 256 | T5-base + LR image, `p_uncond=0.1` |

Total ≈ **120 M params**. Sampler: DDPM, T=1000 denoising steps.

## Files

| File | Size | What it is |
| --- | --- | --- |
| `ckpt_sr_ep650_step89050.pt` | ~1.9 GB | Merged cascade weights (LR base + SR) |
| `samples/` | ~2 MB | 16 demo PNGs at 256² + 4×4 grid |
| `captions.txt` | 72 KB | 1093 RSICD-test captions matching the demo and FID PNGs |
| `fid_result.json` | — | Headline FID (Inception feature=2048) |
| `fid_result_f768.json` | — | Cross-comparison FID (feature=768) |
| `clip_result.json` | — | OpenAI CLIP ViT-B/32 score + shuffled-baseline null |

## Usage

```bash
git clone https://github.com/asebaq/rsdiff
cd rsdiff
uv venv && source .venv/bin/activate
uv pip install -e ".[dev,eval]"

# pull the checkpoint
hf download asebaq/rsdiff-sr-cascade-ep650 ckpt_sr_ep650_step89050.pt -o legacy/DDPM/ckpts/

# sample 16 captions from the RSICD test split
python legacy/DDPM/sample_grid.py \
  --log_dir legacy/DDPM/logs/full_sr_gdm \
  --data_root data/RSICD_optimal \
  --ckpt legacy/DDPM/ckpts/ckpt_sr_ep650_step89050.pt \
  --n 16 --cols 4 --batch 2 --cond_scale 5.0 \
  --img_sz 128 --sr_sz 256 --ts 1000 \
  --sr --split test --seed 17
```

A `diffusers`-native sampling path is on the project roadmap; for now the
bundled cascade runner (`legacy/`) loads this checkpoint directly.

## Training data

[RSICD](https://huggingface.co/datasets/arampacha/rsicd) — 10 921 paired
satellite images and natural-language captions, official 8/1/1
train/val/test split (1093 test). At training time the first caption per
image (`sent1`) is used as the conditioning text.

## Intended use & limitations

**Intended use.** Research artefact for studying small-scale text-to-RS
generation. Useful as a baseline for new remote-sensing diffusion work
and as a starting point for downstream tasks (augmentation, change-
detection priors).

**Out of scope.**

- Operational or commercial remote-sensing imagery synthesis — visual
  fidelity is well below modern web-scale models.
- Generating imagery intended to be mistaken for real satellite data.
- Anything safety-critical (disaster response, surveillance, defence).

**Known limitations.**

- **Overfit drift past SR ep650.** Validation FID climbs slightly after
  the bowl (see [REPORT.md §4](https://github.com/asebaq/rsdiff/blob/main/docs/REPORT.md)).
  No augmentation or weight decay; the train set is small (10 921 images).
- **Single-caption conditioning.** RSICD provides 5 captions per image;
  this run uses only the first.
- **Pixel-space cascade.** Slower at inference than a latent-diffusion
  port; a latent-space rewrite is on the project roadmap.
- **No memorisation probe.** Partial training-set memorisation is not
  ruled out — pHash audit is on the roadmap.

## License

Apache 2.0 — see [`LICENSE`](https://github.com/asebaq/rsdiff/blob/main/LICENSE).

## Citation

```bibtex
@article{sebaq2024rsdiff,
  title   = {RSDiff: remote sensing image generation from text using diffusion model},
  author  = {Sebaq, Ahmad and ElHelw, Mohamed},
  journal = {Neural Computing and Applications},
  volume  = {36},
  number  = {36},
  pages   = {23103--23111},
  year    = {2024},
  doi     = {10.1007/s00521-024-10363-3}
}
```