BiliSakura commited on
Commit
596f048
·
verified ·
1 Parent(s): f8f058d

Update all files for DiffusionSat-Single-512

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ library_name: diffusers
4
+ pipeline_tag: text-to-image
5
+ tags:
6
+ - satellite
7
+ - controlnet
8
+ - diffusers
9
+ - text-to-image
10
+ ---
11
+
12
+ # DiffusionSat Custom Pipelines
13
+
14
+ Custom community pipelines for loading DiffusionSat checkpoints directly with `diffusers.DiffusionPipeline.from_pretrained()`.
15
+
16
+ > See [Diffusers Community Pipeline Documentation](https://huggingface.co/docs/diffusers/using-diffusers/custom_pipeline_overview)
17
+
18
+ ## Model Index
19
+
20
+ `model_index.json` is set to the default text-to-image pipeline (`DiffusionSatPipeline`) so `DiffusionPipeline.from_pretrained()` works out of the box. The ControlNet variant is loaded via `custom_pipeline` plus the `controlnet` subfolder, as shown below.
21
+
22
+ ## Available Pipelines
23
+
24
+ This directory contains two custom pipelines:
25
+
26
+ 1. **`pipeline_diffusionsat.py`**: Standard text-to-image pipeline with DiffusionSat metadata support.
27
+ 2. **`pipeline_diffusionsat_controlnet.py`**: ControlNet pipeline with DiffusionSat metadata and conditional metadata support.
28
+
29
+ ## Setup
30
+
31
+ The checkpoint folder (`ckpt/diffusionsat/`) should contain the standard diffusers components (unet, vae, scheduler, etc.). You can reference these pipeline files directly from this directory or copy them to your checkpoint folder.
32
+
33
+ ## Usage
34
+
35
+ ### 1. Text-to-Image Pipeline
36
+
37
+ Use `pipeline_diffusionsat.py` for standard generation.
38
+
39
+ ```python
40
+ import torch
41
+ from diffusers import DiffusionPipeline
42
+
43
+ # Load pipeline
44
+ pipe = DiffusionPipeline.from_pretrained(
45
+ "path/to/ckpt/diffusionsat",
46
+ custom_pipeline="./custom_pipelines/pipeline_diffusionsat.py", # Path to this file
47
+ torch_dtype=torch.float16,
48
+ trust_remote_code=True,
49
+ )
50
+ pipe = pipe.to("cuda")
51
+
52
+ # Optional: Metadata (normalized lat, lon, timestamp, GSD, etc.)
53
+ # metadata = [0.5, -0.3, 0.7, 0.2, 0.1, 0.0, 0.5]
54
+
55
+ # Generate
56
+ image = pipe(
57
+ "satellite image of farmland",
58
+ metadata=None, # Optional
59
+ num_inference_steps=30,
60
+ ).images[0]
61
+ ```
62
+
63
+ ### 2. ControlNet Pipeline
64
+
65
+ Use `pipeline_diffusionsat_controlnet.py` for ControlNet generation.
66
+
67
+ ```python
68
+ import torch
69
+ from diffusers import DiffusionPipeline, ControlNetModel
70
+ from diffusers.utils import load_image
71
+
72
+ # 1. Load ControlNet
73
+ controlnet = ControlNetModel.from_pretrained(
74
+ "path/to/ckpt/diffusionsat/controlnet",
75
+ torch_dtype=torch.float16
76
+ )
77
+
78
+ # 2. Load Pipeline with ControlNet
79
+ pipe = DiffusionPipeline.from_pretrained(
80
+ "path/to/ckpt/diffusionsat",
81
+ controlnet=controlnet,
82
+ custom_pipeline="./custom_pipelines/pipeline_diffusionsat_controlnet.py", # Path to this file
83
+ torch_dtype=torch.float16,
84
+ trust_remote_code=True,
85
+ )
86
+ pipe = pipe.to("cuda")
87
+
88
+ # 3. Prepare Control Image
89
+ control_image = load_image("path/to/conditioning_image.png")
90
+
91
+ # 4. Generate
92
+ # metadata: Target image metadata (optional)
93
+ # cond_metadata: Conditioning image metadata (optional)
94
+
95
+ image = pipe(
96
+ "satellite image of farmland",
97
+ image=control_image,
98
+ metadata=None,
99
+ cond_metadata=None,
100
+ num_inference_steps=30,
101
+ ).images[0]
102
+ ```