README.md

---
license: apache-2.0
base_model: stabilityai/stable-diffusion-xl-base-1.0
tags:
  - controlnet
  - lora
  - stable-diffusion-xl
  - text-to-image
  - image-to-image
  - remote-sensing
  - satellite-imagery
  - synthetic-data
  - geospatial
pipeline_tag: text-to-image
language:
  - en
---

# ConLoRA-Sat

**Parameter-Efficient Synthetic Satellite Imagery Generation for Data-Scarce Humanitarian Contexts**

ConLoRA-Sat combines **LoRA** (domain-style adaptation) with **ControlNet** (spatial structure) on a **Stable Diffusion XL 1.0** backbone: one text prompt plus a 5-class land-cover segmentation map generates a realistic 0.5 m satellite image. It targets data-scarce humanitarian use, e.g. UN peacekeeping training, producing fictitious yet geographically plausible terrain rather than identifiable real locations.

```
Sam-Roblex/ConLoRA-Sat/
├── lora/
│   └── satellite_lora.safetensors          ← one file: the LoRA weights
└── controlnet/
    ├── config.json                         ← SDXL ControlNet architecture (required by diffusers)
    └── diffusion_pytorch_model.safetensors ← the ControlNet weights
```

- Trained on **322 curated images** (LoRA) and **4,333 image/segmentation pairs** (ControlNet) from Mogadishu (Somalia) and Addis Ababa (Ethiopia). About **24 A6000-equivalent GPU-hours** total. Runs in 16-20 GB inference VRAM.

## Usage

Both components load from this repo via subfolders:

```python
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
import torch

pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=ControlNetModel.from_pretrained(
        "Sam-Roblex/ConLoRA-Sat", subfolder="controlnet", torch_dtype=torch.float16),
    torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("cuda")

pipe.load_lora_weights("Sam-Roblex/ConLoRA-Sat",
                       weight_name="lora/satellite_lora.safetensors")

image = pipe(
    "satellite imagery, aerial view, Mixed-use urban settlement with moderate building density",
    image="seg_map.png",      # 5-class map, 1024×1024
    num_inference_steps=50,
    guidance_scale=7.5,
    controlnet_conditioning_scale=1.0,
).images[0]
```

Notes:
- LoRA trigger phrase is **"satellite imagery, aerial view"**, prepended to a descriptive caption.
- On water-dominated maps lower `controlnet_conditioning_scale` toward **0.8** for smoother coastlines.
- The two components are trained separately and composed at inference, so either can be reused independently.

## Land-cover class legend

| Class      | RGB      |
|------------|----------|
| Bareland   | `#D2B48C` |
| Vegetation | `#228B22` |
| Roads      | `#404040` |
| Buildings  | `#B22222` |
| Water      | `#1E90FF` |

## Model details

- **Backbone**: SDXL 1.0 Base, frozen throughout.
- **LoRA** (`lora/`): cross-attention query/value projections, rank 32, α 16. Prodigy optimizer, Min-SNR weighting (γ 5). ~4,000 steps, ~2 GPU-hours.
- **ControlNet** (`controlnet/`): trainable copy of SDXL's U-Net encoder path, injected via zero-initialised 1×1 convolutions. AdamW (lr 5e-6, 500-step warm-up), DDPM schedule. All 4,333 pairs, ~22 GPU-hours.

## Results

| Config | FID | LPIPS | CMMD | CLIPScore |
|--------|-----|-------|------|-----------|
| LoRA only | 110.98 | 0.640 | 0.034 | 0.249 |
| ControlNet only | 106.44 | 0.490 | 0.063 | 0.246 |
| **ConLoRA-Sat** | **100.88** | **0.474** | 0.061 | 0.248 |

Downstream (DeepLabV3+/ResNet-50): a segmenter trained only on synthetic imagery retains **90.1 %** of a real-data baseline; 200 real + 600 synthetic recovers **92.5 %**. In a 53-expert perceptual study, synthetic images were misclassified as real in **20.8 %** of trials.

## Train / evaluate / cite
Training scripts, evaluation code, and dataset documentation live in the companion GitHub repository. Dataset: [Zenodo: 10.5281/zenodo.22747581](https://doi.org/10.5281/zenodo.22747581) (CC BY-NC 4.0). Paper: *ConLoRA-Sat: Parameter-Efficient Synthetic Satellite Imagery Generation for Data-Scarce Humanitarian Contexts*, EarthArXiv preprint (under review).

```bibtex
@misc{qureshi2026conlorasat,
  title  = {ConLoRA-Sat: Parameter-Efficient Synthetic Satellite Imagery
            Generation for Data-Scarce Humanitarian Contexts},
  author = {Qureshi, Seyed Amir Muhammad and Granell, Carlos and Casteleyn, Sven},
  year   = {2026},
  note   = {EarthArXiv preprint; under review}
}
```

**License**: weights Apache-2.0; base model CreativeML OpenRAIL++-M. Training dataset CC BY-NC 4.0. Generates fictitious terrain, not identifiable real locations.