README.md

---
license: cc-by-nc-sa-4.0
language:
  - en
library_name: diffusers
pipeline_tag: text-to-image
tags:
  - sar
  - synthetic-aperture-radar
  - remote-sensing
  - stable-diffusion
  - image-generation
  - synthetic-data
  - ship-detection
  - earth-observation
  - capella
---

# HR-SAR Stable Diffusion

## Overview

Fine-tuned **Stable Diffusion 1.5** backbone for generating synthetic high-resolution **SAR** amplitude images from text prompts.  
Published alongside the upcoming paper **"Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations"** (Hochstuhl et al., 2026; accepted for GCPR conference 2026).

---

## Database
Fine-tuning was conducted on a SAR–text dataset compiled from high-resolution **Capella (X-band)** imagery originating from the [SpaceNet6 challenge dataset](https://spacenet.ai/sn6-challenge/), which covers the Rotterdam harbor area. The dataset contains geocoded, fully polarimetric Ground Range Detected (GRD) amplitude images in dB-scale with a ground sample distance of 0.5 m, along with co-registered optical WorldView-2 imagery (RGB, 0.5 m GSD). To create SAR–text pairs, the imagery was tiled into **512 × 512** patches (~256 × 256 m), and text captions were generated from the corresponding optical patches using a BLIP model fine-tuned on remote sensing imagery ([BLIP_RSCID](https://huggingface.co/Gurveer05/blip-image-captioning-base-rscid-finetuned)).

---

## Architecture

| Component | Details |
|---|---|
| Base model | Stable Diffusion 1.5 |
| VAE decoder | Adapted to **single-channel** output |
| Text encoder | Fine-tuned with a **LoRA adapter** (`adapter_text_encoder/`) |
| Output | Single-channel float32 SAR amplitude image |
---

## Repository Structure

```text
HR-SAR-StableDiffusion/
├── model_index.json
├── unet/
├── vae/                        # Modified: single-channel conv_out
├── text_encoder/
├── tokenizer/
├── scheduler/
├── feature_extractor/
└── adapter_text_encoder/       # LoRA adapter for text encoder
    ├── adapter_config.json
    └── adapter_model.safetensors
```

---

## Usage
This model can be used directly with the 🤗 [diffusers](https://github.com/huggingface/diffusers) pipeline.
The LoRA adapter for the text encoder is included in this repository and must be loaded manually after initialising the pipeline.

**Requirements:**
- `torch` (CUDA recommended)
- `diffusers`
- `peft`

### Text-to-Image Generation

```python
import torch
from diffusers import StableDiffusionPipeline
from peft import PeftModel

MODEL_ID = "sylviaHoch/HR-SAR-StableDiffusion"

# Load pipeline
pipeline = StableDiffusionPipeline.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16
).to("cuda")

# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
    pipeline.text_encoder,
    MODEL_ID,
    subfolder="adapter_text_encoder"
)

# Generate
image = pipeline(
    "An aerial view of oil tanks.",
    num_inference_steps=50,
    guidance_scale=3.0,
    output_type="pt"
).images

```

>image is a torch.Tensor of shape (N, C, H, W) with values in the range [0, 1] and dtype float32, where N is the batch size (here 1), C the number of channels (here 1, grayscale), and H and W the native model resolution (512 x 512).
---

## Intended Use

- Synthetic SAR data generation.
- Research on generative models for remote sensing.

## Limitations

- Tuned specifically to **Capella** sensor characteristics based on multilooked, geocoded images in VH polarization; transferability to other sensors and imaging properties is very limited.
- Generated images are synthetic and may not fully capture all real SAR image properties.

---

## Citation

If you use this model, please cite:

```bibtex
@inproceedings{sar_diffusion_gcpr2026,
  title     = {Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  year      = {2026},
  note      = {accepted, to be published},
  authors   = {}
}
```

---

## License

This model is released under **CC BY-NC-SA 4.0**.  
Commercial use is not permitted. Derivatives must be shared under the same license.  
See [LICENSE](https://creativecommons.org/licenses/by-nc-sa/4.0/) for details.