README.md

---
license: cc-by-nc-sa-4.0
language:
  - en
library_name: diffusers
pipeline_tag: text-to-image
tags:
  - sar
  - synthetic-aperture-radar
  - remote-sensing
  - stable-diffusion
  - controlnet
  - image-generation
  - synthetic-data
  - ship-detection
  - earth-observation
  - sentinel-1
---

# SAR ControlNet

## Overview

**ControlNet** for spatially controlled synthesis of **Sentinel-1 SAR** images.  
Published alongside the upcoming paper **"Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations"** (Hochstuhl et al., 2026; accepted for GCPR conference 2026).

This model must be used together with the corresponding Stable Diffusion backbone:  
[`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion)

Spatial control is provided via three-channel one-hot masks (water / ship / land), enabling precise placement of ships in the generated SAR images.

---

## Database
The ControlNet was trained on the training set of **OpenSARShip Ship Detection Dataset (OSSDD)** [[dataset]](https://huggingface.co/datasets/sylviaHoch/OpenSARShip-Ship-Detection-Dataset), a Sentinel-1 dataset with 15,161 patches (700 × 700 px) and 64,640 ships, providing rotated bounding boxes (RBBs) and land–water masks. Text captions used during ControlNet training depend on land coverage: open-water patches are captioned *"An aerial view of ships on open water."*, patches with land (> 5 % of pixels) *"An aerial view of a coastal area with ships on the water."* During training, 512 × 512 crops are extracted, amplitude values converted to dB and linearly mapped to `[-1, 1]`.

## Architecture

| Component | Details |
|---|---|
| Type | ControlNet (based on SD 1.5 UNet) |
| Control input — ship + land | Three-channel one-hot mask `(3, H, W)` — ch0: background, ch1: ship, ch2: land |
| Required backbone | [`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion) |

---

## Repository Structure

```text
SAR-ControlNet/
├── config.json
└── diffusion_pytorch_model.safetensors
```

---

## Usage

This model can be used directly with the 🤗 [diffusers](https://github.com/huggingface/diffusers) ControlNet pipeline. It requires the corresponding Stable Diffusion backbone ([`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion)).

**Requirements:**
- `torch`
- `diffusers`
- `peft`

**Control input:** The control image must be a three-channel, one-hot encoded mask of size **512 × 512**, i.e. a tensor of shape `(3, 512, 512)` with values in `{0, 1}`. Each pixel is assigned to exactly one class across the three channels:

- **Channel 0:** water
- **Channel 1:** ship (represented as rotated bounding box/ minimal enclosing rectangle)
- **Channel 2:** land

**Output:** `image` is a `torch.Tensor` of shape `(N, C, H, W)` with values in the range `[0, 1]` and dtype `float32`, where `N` is the batch size (here `1`), `C` the number of channels (here `1`, single-channel SAR amplitude), and `H` and `W` the native model resolution.

```python
import torch
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, UniPCMultistepScheduler
from peft import PeftModel

SD_MODEL_ID = "sylviaHoch/SAR-StableDiffusion"
CN_MODEL_ID = "sylviaHoch/SAR-ControlNet"

# control_image: torch.Tensor of shape (3, 512, 512), one-hot encoded
#   channel 0: water, channel 1: ship, channel 2: land

controlnet = ControlNetModel.from_pretrained(CN_MODEL_ID, torch_dtype=torch.float16)

pipeline = StableDiffusionControlNetPipeline.from_pretrained(
    SD_MODEL_ID,
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
    pipeline.text_encoder,
    SD_MODEL_ID,
    subfolder="adapter_text_encoder"
)

pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)

# Generate
image = pipeline(
    "An aerial view of ships on open water.",
    control_image,
    num_inference_steps=50,
    guidance_scale=7.5,
    controlnet_conditioning_scale=1.0,
    output_type="pt"
).images
```

---

## Intended Use

- Spatially controlled synthesis of Sentinel-1 SAR training data.
- Ship placement guided by spatial annotations.
- Training data generation or data augmentation for ship detection models.

## Limitations

- Tuned specifically to **Sentinel-1** sensor characteristics (in particular Sentinel-1 GRD products in VH polarization).
- Training scenes are limited to harbour, coastal, and open water scenes.
- Generated images are synthetic and may not fully capture all real SAR image properties.

---

<!-- ## Citation

If you use this model, please cite:

```bibtex
@inproceedings{sar_diffusion_gcpr2026,
  title     = {Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  year      = {2026},
  note      = {accepted, to be published},
  authors   = {}
}
```

--- -->

## License

This model is released under **CC BY-NC-SA 4.0**.  
Commercial use is not permitted. Derivatives must be shared under the same license.  
See [LICENSE](https://creativecommons.org/licenses/by-nc-sa/4.0/) for details.
```