ConLoRA-Sat

Parameter-Efficient Synthetic Satellite Imagery Generation for Data-Scarce Humanitarian Contexts

ConLoRA-Sat combines LoRA (domain-style adaptation) with ControlNet (spatial structure) on a Stable Diffusion XL 1.0 backbone: one text prompt plus a 5-class land-cover segmentation map generates a realistic 0.5 m satellite image. It targets data-scarce humanitarian use, e.g. UN peacekeeping training, producing fictitious yet geographically plausible terrain rather than identifiable real locations.

Sam-Roblex/ConLoRA-Sat/
├── lora/
│   └── satellite_lora.safetensors          ← one file: the LoRA weights
└── controlnet/
    ├── config.json                         ← SDXL ControlNet architecture (required by diffusers)
    └── diffusion_pytorch_model.safetensors ← the ControlNet weights

• Trained on 322 curated images (LoRA) and 4,333 image/segmentation pairs (ControlNet) from Mogadishu (Somalia) and Addis Ababa (Ethiopia). About 24 A6000-equivalent GPU-hours total. Runs in 16-20 GB inference VRAM.

Usage

Both components load from this repo via subfolders:

from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
import torch

pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=ControlNetModel.from_pretrained(
        "Sam-Roblex/ConLoRA-Sat", subfolder="controlnet", torch_dtype=torch.float16),
    torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("cuda")

pipe.load_lora_weights("Sam-Roblex/ConLoRA-Sat",
                       weight_name="lora/satellite_lora.safetensors")

image = pipe(
    "satellite imagery, aerial view, Mixed-use urban settlement with moderate building density",
    image="seg_map.png",      # 5-class map, 1024×1024
    num_inference_steps=50,
    guidance_scale=7.5,
    controlnet_conditioning_scale=1.0,
).images[0]

Notes:

• LoRA trigger phrase is "satellite imagery, aerial view", prepended to a descriptive caption.
• On water-dominated maps lower controlnet_conditioning_scale toward 0.8 for smoother coastlines.
• The two components are trained separately and composed at inference, so either can be reused independently.

Land-cover class legend

Class	RGB
Bareland	#D2B48C
Vegetation	#228B22
Roads	#404040
Buildings	#B22222
Water	#1E90FF

Model details

• Backbone: SDXL 1.0 Base, frozen throughout.
• LoRA (lora/): cross-attention query/value projections, rank 32, α 16. Prodigy optimizer, Min-SNR weighting (γ 5). ~4,000 steps, ~2 GPU-hours.
• ControlNet (controlnet/): trainable copy of SDXL's U-Net encoder path, injected via zero-initialised 1×1 convolutions. AdamW (lr 5e-6, 500-step warm-up), DDPM schedule. All 4,333 pairs, ~22 GPU-hours.

Results

Config	FID	LPIPS	CMMD	CLIPScore
LoRA only	110.98	0.640	0.034	0.249
ControlNet only	106.44	0.490	0.063	0.246
ConLoRA-Sat	100.88	0.474	0.061	0.248

Downstream (DeepLabV3+/ResNet-50): a segmenter trained only on synthetic imagery retains 90.1 % of a real-data baseline; 200 real + 600 synthetic recovers 92.5 %. In a 53-expert perceptual study, synthetic images were misclassified as real in 20.8 % of trials.

Train / evaluate / cite

Training scripts, evaluation code, and dataset documentation live in the companion GitHub repository. Dataset: Zenodo: 10.5281/zenodo.22747581 (CC BY-NC 4.0). Paper: ConLoRA-Sat: Parameter-Efficient Synthetic Satellite Imagery Generation for Data-Scarce Humanitarian Contexts, EarthArXiv preprint (under review).

@misc{qureshi2026conlorasat,
  title  = {ConLoRA-Sat: Parameter-Efficient Synthetic Satellite Imagery
            Generation for Data-Scarce Humanitarian Contexts},
  author = {Qureshi, Seyed Amir Muhammad and Granell, Carlos and Casteleyn, Sven},
  year   = {2026},
  note   = {EarthArXiv preprint; under review}
}

License: weights Apache-2.0; base model CreativeML OpenRAIL++-M. Training dataset CC BY-NC 4.0. Generates fictitious terrain, not identifiable real locations.