File size: 5,410 Bytes
f71bf9c cfd31dc f71bf9c cfd31dc f71bf9c cfd31dc f71bf9c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | ---
license: cc-by-nc-sa-4.0
language:
- en
library_name: diffusers
pipeline_tag: text-to-image
tags:
- sar
- synthetic-aperture-radar
- remote-sensing
- stable-diffusion
- controlnet
- image-generation
- synthetic-data
- ship-detection
- earth-observation
- sentinel-1
---
# SAR ControlNet
## Overview
**ControlNet** for spatially controlled synthesis of **Sentinel-1 SAR** images.
Published alongside the upcoming paper **"Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations"** (Hochstuhl et al., 2026; accepted for GCPR conference 2026).
This model must be used together with the corresponding Stable Diffusion backbone:
[`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion)
Spatial control is provided via three-channel one-hot masks (water / ship / land), enabling precise placement of ships in the generated SAR images.
---
## Database
The ControlNet was trained on the training set of **OpenSARShip Ship Detection Dataset (OSSDD)** [[dataset]](https://huggingface.co/datasets/sylviaHoch/OpenSARShip-Ship-Detection-Dataset), a Sentinel-1 dataset with 15,161 patches (700 × 700 px) and 64,640 ships, providing rotated bounding boxes (RBBs) and land–water masks. Text captions used during ControlNet training depend on land coverage: open-water patches are captioned *"An aerial view of ships on open water."*, patches with land (> 5 % of pixels) *"An aerial view of a coastal area with ships on the water."* During training, 512 × 512 crops are extracted, amplitude values converted to dB and linearly mapped to `[-1, 1]`.
## Architecture
| Component | Details |
|---|---|
| Type | ControlNet (based on SD 1.5 UNet) |
| Control input — ship + land | Three-channel one-hot mask `(3, H, W)` — ch0: background, ch1: ship, ch2: land |
| Required backbone | [`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion) |
---
## Repository Structure
```text
SAR-ControlNet/
├── config.json
└── diffusion_pytorch_model.safetensors
```
---
## Usage
This model can be used directly with the 🤗 [diffusers](https://github.com/huggingface/diffusers) ControlNet pipeline. It requires the corresponding Stable Diffusion backbone ([`sylviaHoch/SAR-StableDiffusion`](https://huggingface.co/sylviaHoch/SAR-StableDiffusion)).
**Requirements:**
- `torch`
- `diffusers`
- `peft`
**Control input:** The control image must be a three-channel, one-hot encoded mask of size **512 × 512**, i.e. a tensor of shape `(3, 512, 512)` with values in `{0, 1}`. Each pixel is assigned to exactly one class across the three channels:
- **Channel 0:** water
- **Channel 1:** ship (represented as rotated bounding box/ minimal enclosing rectangle)
- **Channel 2:** land
**Output:** `image` is a `torch.Tensor` of shape `(N, C, H, W)` with values in the range `[0, 1]` and dtype `float32`, where `N` is the batch size (here `1`), `C` the number of channels (here `1`, single-channel SAR amplitude), and `H` and `W` the native model resolution.
```python
import torch
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, UniPCMultistepScheduler
from peft import PeftModel
SD_MODEL_ID = "sylviaHoch/SAR-StableDiffusion"
CN_MODEL_ID = "sylviaHoch/SAR-ControlNet"
# control_image: torch.Tensor of shape (3, 512, 512), one-hot encoded
# channel 0: water, channel 1: ship, channel 2: land
controlnet = ControlNetModel.from_pretrained(CN_MODEL_ID, torch_dtype=torch.float16)
pipeline = StableDiffusionControlNetPipeline.from_pretrained(
SD_MODEL_ID,
controlnet=controlnet,
torch_dtype=torch.float16
).to("cuda")
# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
pipeline.text_encoder,
SD_MODEL_ID,
subfolder="adapter_text_encoder"
)
pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)
# Generate
image = pipeline(
"An aerial view of ships on open water.",
control_image,
num_inference_steps=50,
guidance_scale=7.5,
controlnet_conditioning_scale=1.0,
output_type="pt"
).images
```
---
## Intended Use
- Spatially controlled synthesis of Sentinel-1 SAR training data.
- Ship placement guided by spatial annotations.
- Training data generation or data augmentation for ship detection models.
## Limitations
- Tuned specifically to **Sentinel-1** sensor characteristics (in particular Sentinel-1 GRD products in VH polarization).
- Training scenes are limited to harbour, coastal, and open water scenes.
- Generated images are synthetic and may not fully capture all real SAR image properties.
---
<!-- ## Citation
If you use this model, please cite:
```bibtex
@inproceedings{sar_diffusion_gcpr2026,
title = {Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations},
booktitle = {German Conference on Pattern Recognition (GCPR)},
year = {2026},
note = {accepted, to be published},
authors = {}
}
```
--- -->
## License
This model is released under **CC BY-NC-SA 4.0**.
Commercial use is not permitted. Derivatives must be shared under the same license.
See [LICENSE](https://creativecommons.org/licenses/by-nc-sa/4.0/) for details.
``` |