metadata
license: cc-by-nc-sa-4.0
library_name: diffusers
pipeline_tag: text-to-image
base_model: lcybuaa/Text2Earth
tags:
- controlnet
- remote-sensing
- xbd
- building-damage
Text2Earth ControlNet for xBD
ControlNet fine-tuned on xBD post-event imagery. It takes a building-damage mask and samples a matching satellite image from Text2Earth.
The ControlNet is the only trained module. Text2Earth (UNet, VAE, text encoder) stays frozen.
Conditioning
| Item | Value |
|---|---|
| Base model | lcybuaa/Text2Earth |
| Conditioning | 3-channel binary damage mask, RGB order, values in {0, 1} |
| Prompt | 18_GOOGLE_LEVEL_a high resolution satellite image |
| Scheduler | Euler discrete |
| Steps / guidance | 50 / 1.0 |
Binary channels follow the xBD hues: background black, no-damage green, minor blue, major yellow, destroy red, un-classified white.
Usage
import torch
from diffusers import (
ControlNetModel,
EulerDiscreteScheduler,
StableDiffusionControlNetPipeline,
)
from PIL import Image
repo = "BiliSakura/Text2Earth-ControlNet-xBD"
base = "lcybuaa/Text2Earth"
dtype = torch.float16
controlnet = ControlNetModel.from_pretrained(
repo, subfolder="controlnet", torch_dtype=dtype
)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
base,
controlnet=controlnet,
scheduler=EulerDiscreteScheduler.from_pretrained(base, subfolder="scheduler"),
torch_dtype=dtype,
safety_checker=None,
).to("cuda")
image = pipe(
"18_GOOGLE_LEVEL_a high resolution satellite image",
image=Image.open("mask.png"),
num_inference_steps=50,
guidance_scale=1.0,
controlnet_conditioning_scale=1.0,
).images[0]
image.save("post_event.png")
Weights are under controlnet/.
License
CC BY-NC-SA 4.0, the license of the xBD dataset these weights were trained on.