sylviaHoch/HR-SAR-StableDiffusion

HR-SAR Stable Diffusion

Overview

Fine-tuned Stable Diffusion 1.5 backbone for generating synthetic high-resolution SAR amplitude images from text prompts.
Published alongside the upcoming paper "Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations" (Hochstuhl et al., 2026; accepted for GCPR conference 2026).

---

Database

Fine-tuning was conducted on a SAR–text dataset compiled from high-resolution Capella (X-band) imagery originating from the SpaceNet6 challenge dataset, which covers the Rotterdam harbor area. The dataset contains geocoded, fully polarimetric Ground Range Detected (GRD) amplitude images in dB-scale with a ground sample distance of 0.5 m, along with co-registered optical WorldView-2 imagery (RGB, 0.5 m GSD). To create SAR–text pairs, the imagery was tiled into 512 × 512 patches (~256 × 256 m), and text captions were generated from the corresponding optical patches using a BLIP model fine-tuned on remote sensing imagery (BLIP_RSCID).

---

Architecture

Component	Details
Base model	Stable Diffusion 1.5
VAE decoder	Adapted to single-channel output
Text encoder	Fine-tuned with a LoRA adapter (adapter_text_encoder/)
Output	Single-channel float32 SAR amplitude image

---

Repository Structure

HR-SAR-StableDiffusion/
├── model_index.json
├── unet/
├── vae/                        # Modified: single-channel conv_out
├── text_encoder/
├── tokenizer/
├── scheduler/
├── feature_extractor/
└── adapter_text_encoder/       # LoRA adapter for text encoder
    ├── adapter_config.json
    └── adapter_model.safetensors

---

Usage

This model can be used directly with the 🤗 diffusers pipeline. The LoRA adapter for the text encoder is included in this repository and must be loaded manually after initialising the pipeline.

Requirements:

• torch (CUDA recommended)
• diffusers
• peft

Text-to-Image Generation

import torch
from diffusers import StableDiffusionPipeline
from peft import PeftModel

MODEL_ID = "sylviaHoch/HR-SAR-StableDiffusion"

# Load pipeline
pipeline = StableDiffusionPipeline.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16
).to("cuda")

# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
    pipeline.text_encoder,
    MODEL_ID,
    subfolder="adapter_text_encoder"
)

# Generate
image = pipeline(
    "An aerial view of oil tanks.",
    num_inference_steps=50,
    guidance_scale=3.0,
    output_type="pt"
).images

image is a torch.Tensor of shape (N, C, H, W) with values in the range [0, 1] and dtype float32, where N is the batch size (here 1), C the number of channels (here 1, grayscale), and H and W the native model resolution (512 x 512).

---

Intended Use

• Synthetic SAR data generation.
• Research on generative models for remote sensing.

Limitations

• Tuned specifically to Capella sensor characteristics based on multilooked, geocoded images in VH polarization; transferability to other sensors and imaging properties is very limited.
• Generated images are synthetic and may not fully capture all real SAR image properties.

---

Citation

If you use this model, please cite:

@inproceedings{sar_diffusion_gcpr2026,
  title     = {Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  year      = {2026},
  note      = {accepted, to be published},
  authors   = {}
}

---

License

This model is released under CC BY-NC-SA 4.0.
Commercial use is not permitted. Derivatives must be shared under the same license.
See LICENSE for details.