File size: 2,195 Bytes
05f3422 0643d49 05f3422 73d6240 05f3422 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
license: apache-2.0
library_name: diffusers
pipeline_tag: image-to-image
tags:
- controlnet
- remote-sensing
- openstreetmap
widget:
- src: demo_images/input.jpeg
prompt: convert this openstreetmap into its satellite view
output:
url: demo_images/output.jpeg
---
> [!WARNING] we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via bili_sakura@zju.edu.cn
# ControlEarth
ControlNet model conditioned on OpenStreetMaps (OSM) to generate the corresponding satellite images.
Trained on the region of the Central Belt.
## Repo structure
This repo is self-contained and includes:
- **controlnet/** — ControlNet weights (OSM → satellite)
- **text_encoder/**, **unet/**, **vae/**, **scheduler/**, **tokenizer/** — Stable Diffusion v1-5 base
- **demo_images/** — Placeholder for input OSM images
- **inference_demo.py** — Full diffusers inference script (uses only this repo, no external downloads)
## Dataset used for training
The dataset used for the training procedure is the
[WorldImagery Clarity dataset](https://www.arcgis.com/home/item.html?id=ab399b847323487dba26809bf11ea91a).
The code for the dataset construction can be accessed in https://github.com/miquel-espinosa/map-sat.
## Usage
```bash
# From the repo root
python inference_demo.py
```
Or load programmatically:
```python
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
import torch
repo = "/path/to/controlearth" # or "." when run from repo root
controlnet = ControlNetModel.from_pretrained(f"{repo}/controlnet", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
repo, controlnet=controlnet, torch_dtype=torch.float16,
safety_checker=None, requires_safety_checker=False
)
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
pipe.enable_model_cpu_offload()
image = pipe("convert this openstreetmap into its satellite view", num_inference_steps=50, image=control_image).images[0]
```
|