README.md

metadata

license: cc-by-nc-sa-4.0
language:
  - en
library_name: diffusers
pipeline_tag: text-to-image
tags:
  - sar
  - synthetic-aperture-radar
  - remote-sensing
  - stable-diffusion
  - controlnet
  - image-generation
  - synthetic-data
  - ship-detection
  - earth-observation
  - sentinel-1

SAR ControlNet

Overview

ControlNet for spatially controlled synthesis of Sentinel-1 SAR images.
Published alongside the upcoming paper "Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations" (Hochstuhl et al., 2026; accepted for GCPR conference 2026).

This model must be used together with the corresponding Stable Diffusion backbone:
sylviaHoch/SAR-StableDiffusion

Spatial control is provided via three-channel one-hot masks (water / ship / land), enabling precise placement of ships in the generated SAR images.

---

Database

The ControlNet was trained on the training set of OpenSARShip Ship Detection Dataset (OSSDD) [dataset], a Sentinel-1 dataset with 15,161 patches (700 × 700 px) and 64,640 ships, providing rotated bounding boxes (RBBs) and land–water masks. Text captions used during ControlNet training depend on land coverage: open-water patches are captioned "An aerial view of ships on open water.", patches with land (> 5 % of pixels) "An aerial view of a coastal area with ships on the water." During training, 512 × 512 crops are extracted, amplitude values converted to dB and linearly mapped to [-1, 1].

Architecture

Component	Details
Type	ControlNet (based on SD 1.5 UNet)
Control input — ship + land	Three-channel one-hot mask (3, H, W) — ch0: background, ch1: ship, ch2: land
Required backbone	sylviaHoch/SAR-StableDiffusion

---

Repository Structure

SAR-ControlNet/
├── config.json
└── diffusion_pytorch_model.safetensors

---

Usage

This model can be used directly with the 🤗 diffusers ControlNet pipeline. It requires the corresponding Stable Diffusion backbone (sylviaHoch/SAR-StableDiffusion).

Requirements:

• torch
• diffusers
• peft

Control input: The control image must be a three-channel, one-hot encoded mask of size 512 × 512, i.e. a tensor of shape (3, 512, 512) with values in {0, 1}. Each pixel is assigned to exactly one class across the three channels:

• Channel 0: water
• Channel 1: ship (represented as rotated bounding box/ minimal enclosing rectangle)
• Channel 2: land

Output: image is a torch.Tensor of shape (N, C, H, W) with values in the range [0, 1] and dtype float32, where N is the batch size (here 1), C the number of channels (here 1, single-channel SAR amplitude), and H and W the native model resolution.

import torch
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, UniPCMultistepScheduler
from peft import PeftModel

SD_MODEL_ID = "sylviaHoch/SAR-StableDiffusion"
CN_MODEL_ID = "sylviaHoch/SAR-ControlNet"

# control_image: torch.Tensor of shape (3, 512, 512), one-hot encoded
#   channel 0: water, channel 1: ship, channel 2: land

controlnet = ControlNetModel.from_pretrained(CN_MODEL_ID, torch_dtype=torch.float16)

pipeline = StableDiffusionControlNetPipeline.from_pretrained(
    SD_MODEL_ID,
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
    pipeline.text_encoder,
    SD_MODEL_ID,
    subfolder="adapter_text_encoder"
)

pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)

# Generate
image = pipeline(
    "An aerial view of ships on open water.",
    control_image,
    num_inference_steps=50,
    guidance_scale=7.5,
    controlnet_conditioning_scale=1.0,
    output_type="pt"
).images

---

Intended Use

• Spatially controlled synthesis of Sentinel-1 SAR training data.
• Ship placement guided by spatial annotations.
• Training data generation or data augmentation for ship detection models.

Limitations

• Tuned specifically to Sentinel-1 sensor characteristics (in particular Sentinel-1 GRD products in VH polarization).
• Training scenes are limited to harbour, coastal, and open water scenes.
• Generated images are synthetic and may not fully capture all real SAR image properties.

---

License

This model is released under CC BY-NC-SA 4.0.
Commercial use is not permitted. Derivatives must be shared under the same license.
See LICENSE for details. ```