Add metadata and improve model card structure

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +19 -148
README.md CHANGED
@@ -1,29 +1,28 @@
1
  ---
2
  license: apache-2.0
 
3
  tags:
4
  - Diffusion
5
  - Augmentation
6
  - PromptControlledDiffusion
7
  - semanticsegmentation
8
  - synthetic-data
 
9
  ---
10
- ---
11
- license: apache-2.0
12
- ---
13
  <div align="center">
14
 
15
  # ๐ŸŽจ SyntheticGen
16
 
17
- ### Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation (Not just to add more images, but to add the **right images**)
18
 
19
  *Addressing class imbalance in remote sensing datasets through controlled synthetic generation*
20
 
21
  [![Accepted at IEEE IGARSS 2026](https://img.shields.io/badge/Accepted-IEEE%20IGARSS%202026-1f77b4)](#)
22
- [![arXiv Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2602.04749)
 
23
  [![Live Demo](https://img.shields.io/badge/Live%20Demo-Colab-orange?logo=googlecolab&logoColor=white)](https://colab.research.google.com/drive/11KqBQogdIjwC6UXAGVeD4cfq_VclUC_I?usp=sharing)
24
- [![Hugging Face Weights](https://img.shields.io/badge/Weights-Hugging%20Face-yellow?logo=huggingface&logoColor=black)](https://huggingface.co/buddhi19/SyntheticGen/tree/main)
25
- [![Dataset](https://img.shields.io/badge/Dataset-Google%20Drive-blue)](https://drive.google.com/drive/folders/14cMpLTgvcLdXhRY0kGhFKpDRMvpok90h?usp=sharing)
26
- [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
27
 
28
  </div>
29
 
@@ -31,52 +30,19 @@ license: apache-2.0
31
 
32
  ## ๐ŸŒŸ Overview
33
 
34
- **SyntheticGen** (Official Implementation for the paper https://huggingface.co/papers/2602.04749) tackles the long-tail distribution problem in LoveDA by generating synthetic imagery with *explicit control* over class ratios. You can specify exactly what proportion of each land cover class should appear in the output.
35
-
36
- ## ๐Ÿ”ฅ๐Ÿ”ฅ Updates
37
- - ๐Ÿš€ **Try SyntheticGen in 2 minutes โ€” no setup required** at [**โœจโœจLive Demoโœจโœจ**](https://colab.research.google.com/drive/11KqBQogdIjwC6UXAGVeD4cfq_VclUC_I?usp=sharing)
38
- - ๐Ÿค— Weights Released at [**HuggingFace**](https://huggingface.co/buddhi19/SyntheticGen/tree/main)
39
- - Our paper was accepted to EEE International Geoscience and Remote Sensing Symposium (IGARSS) 2026.
40
 
41
  ### โœจ Highlights
42
- - Two-stage pipeline: ratio-conditioned layout D3PM + ControlNet image synthesis.
43
- - Full or sparse ratio control (e.g., `building:0.4`).
44
- - Config-first workflow for reproducible experiments.
45
 
46
  <div align="center">
47
- <img src="image.png" alt="SyntheticGen Results" width="100%">
48
  </div>
49
 
50
-
51
  ---
52
 
53
- ## โ“ What we try to answer
54
-
55
- ๐Ÿ›ฐ๏ธ **Why is remote-sensing segmentation still difficult, even with strong modern models?**
56
-
57
-
58
- Because the problem is not only in the model โ€” it is also in the data. Some land-cover classes appear again and again, while others are so rare that the model barely gets a chance to learn them. In LoveDA, this becomes even more challenging because the dataset is split into **Urban** and **Rural** domains, each with different scene characteristics and different class distributions.
59
-
60
- โš–๏ธ **So what if we could control the data instead of just accepting it as it is?**
61
-
62
-
63
- That is exactly the idea behind **SyntheticGen**. Instead of using augmentation as a random process, SyntheticGen makes it **controllable**. Users can explicitly specify target class ratios and domain conditions during generation, making it possible to create synthetic samples that are not just more numerous, but more *useful*. This means rare classes can be strengthened deliberately, while still preserving realistic layouts and domain-consistent appearance.
64
-
65
- ๐Ÿง  **What makes SyntheticGen stand out?**
66
-
67
-
68
- Its strength lies in a carefully designed **two-stage pipeline**. First, a **ratio-conditioned discrete diffusion model** generates semantically meaningful layouts. Then, a **ControlNet-guided image synthesis stage** converts those layouts into realistic remote-sensing imagery. By separating **semantic control** from **visual rendering**, the framework achieves something highly valuable: it is both **principled** and **practical**.
69
-
70
- ๐Ÿš€ **Why does that matter beyond this single benchmark?**
71
-
72
-
73
- Because this is not just another generative model for remote sensing. SyntheticGen introduces a targeted augmentation strategy for improving segmentation under **class imbalance** and **domain shift**, and shows that synthetic data can be used not just to add more images, but to add the **right images**.
74
-
75
- ๐ŸŒ **The bigger message**
76
-
77
-
78
- SyntheticGen is a step toward **data-centric remote-sensing segmentation** โ€” a setting where the training distribution is no longer passively accepted, but actively designed. Our paper shows that better segmentation is not only about building better models, but also about building better data.
79
-
80
  ## ๐Ÿš€ Quick Start
81
 
82
  ### Installation
@@ -85,18 +51,9 @@ git clone https://github.com/Buddhi19/SyntheticGen.git
85
  cd SyntheticGen
86
  pip install -r requirements.txt
87
  ```
88
- #### Install Dependencies
89
- ```bash
90
- conda create -n diffusors python=3.10.19 -y
91
- conda activate diffusors
92
-
93
- # official PyTorch Linux + CUDA 12.8 install for v2.10.0
94
- python -m pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128
95
-
96
- python -m pip install -r SyntheticGen/requirements.txt
97
- ```
98
 
99
  ### Generate Your First Synthetic Image
 
100
  ```bash
101
  python src/scripts/sample_pair.py \
102
  --config configs/sample_pair_ckpt40000_building0.4.yaml
@@ -106,7 +63,7 @@ python src/scripts/sample_pair.py \
106
 
107
  ## ๐Ÿ“š Usage
108
 
109
- ### Training Pipeline (Configs)
110
 
111
  **Stage A: Train Layout Generator (D3PM)**
112
  ```bash
@@ -114,27 +71,14 @@ python src/scripts/train_layout_d3pm.py \
114
  --config configs/train_layout_d3pm_masked_sparse_80k.yaml
115
  ```
116
 
117
- **(Optional) Ratio Prior for Sparse Conditioning**
118
- ```bash
119
- python src/scripts/compute_ratio_prior.py \
120
- --config configs/compute_ratio_prior_loveda_train.yaml
121
- ```
122
-
123
  **Stage B: Train Image Generator (ControlNet)**
124
  ```bash
125
  python src/scripts/train_controlnet_ratio.py \
126
  --config configs/train_controlnet_ratio_loveda_1024.yaml
127
  ```
128
 
129
- ### Inference / Sampling (Configs)
130
-
131
- **End-to-end sampling (layout -> image):**
132
- ```bash
133
- python src/scripts/sample_pair.py \
134
- --config configs/sample_pair_ckpt40000_building0.4.yaml
135
- ```
136
-
137
- **Override config parameters via CLI if needed:**
138
  ```bash
139
  python src/scripts/sample_pair.py \
140
  --config configs/sample_pair_ckpt40000_building0.4.yaml \
@@ -144,72 +88,6 @@ python src/scripts/sample_pair.py \
144
 
145
  ---
146
 
147
- ## โš™๏ธ Configuration
148
-
149
- All experiments are driven by YAML/JSON config files in `configs/`.
150
-
151
- | Task | Script | Example Config |
152
- |------|--------|----------------|
153
- | Layout Training | `src/scripts/train_layout_d3pm.py` | `configs/train_layout_d3pm_masked_sparse_80k.yaml` |
154
- | Ratio Prior | `src/scripts/compute_ratio_prior.py` | `configs/compute_ratio_prior_loveda_train.yaml` |
155
- | ControlNet Training | `src/scripts/train_controlnet_ratio.py` | `configs/train_controlnet_ratio_loveda_1024.yaml` |
156
- | Sampling / Inference | `src/scripts/sample_pair.py` | `configs/sample_pair_ckpt40000_building0.4.yaml` |
157
-
158
- **Config tips**
159
- - Examples live in `configs/`.
160
- - To resume training, set `resume_from_checkpoint: "checkpoint-XXXXX"` in your config.
161
- - Dataset roots and domains are centralized in configs; edit once, reuse everywhere.
162
- - CLI flags override config values for quick experiments.
163
-
164
- ---
165
-
166
- ## ๐Ÿ“ Data Format
167
-
168
- ### LoveDA Dataset Structure
169
- ```
170
- LoveDA/
171
- Train/
172
- Train/ # some releases include this extra nesting
173
- Urban/
174
- images_png/
175
- masks_png/
176
- Rural/
177
- images_png/
178
- masks_png/
179
- Urban/
180
- images_png/
181
- masks_png/
182
- Rural/
183
- images_png/
184
- masks_png/
185
- Val/
186
- ...
187
- ```
188
-
189
- ### Generic Dataset Structure
190
- ```
191
- your_dataset/
192
- images/
193
- image_001.png
194
- masks/
195
- image_001.png # label map with matching stem
196
- ```
197
-
198
- ---
199
-
200
- ## ๐Ÿ“ฆ Pre-Generated Datasets
201
-
202
- We provide synthetic datasets used in the paper:
203
- https://drive.google.com/drive/folders/14cMpLTgvcLdXhRY0kGhFKpDRMvpok90h?usp=sharing
204
-
205
- ---
206
-
207
- ## ๐Ÿงพ Outputs
208
- - Checkpoints include `training_config.json` and `class_names.json`.
209
- - Sampling writes `image.png`, `layout.png`, and `metadata.json`.
210
-
211
- ---
212
-
213
  ## ๐Ÿ“„ Citation
214
  ```bibtex
215
  @misc{wijenayake2026mitigating,
@@ -225,14 +103,7 @@ https://drive.google.com/drive/folders/14cMpLTgvcLdXhRY0kGhFKpDRMvpok90h?usp=sha
225
 
226
  ---
227
 
228
- ## ๐Ÿ“ License
229
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
230
-
231
- ---
232
-
233
  ## ๐Ÿ™ Acknowledgments
234
- - LoveDA dataset creators for high-quality annotated remote sensing data
235
- - Hugging Face Diffusers for diffusion model infrastructure
236
- - ControlNet authors for controllable generation
237
-
238
- ---
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: image-to-image
4
  tags:
5
  - Diffusion
6
  - Augmentation
7
  - PromptControlledDiffusion
8
  - semanticsegmentation
9
  - synthetic-data
10
+ - remote-sensing
11
  ---
12
+
 
 
13
  <div align="center">
14
 
15
  # ๐ŸŽจ SyntheticGen
16
 
17
+ ### Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation
18
 
19
  *Addressing class imbalance in remote sensing datasets through controlled synthetic generation*
20
 
21
  [![Accepted at IEEE IGARSS 2026](https://img.shields.io/badge/Accepted-IEEE%20IGARSS%202026-1f77b4)](#)
22
+ [![arXiv Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://huggingface.co/papers/2602.04749)
23
+ [![GitHub Code](https://img.shields.io/badge/Code-GitHub-blue?logo=github)](https://github.com/Buddhi19/SyntheticGen)
24
  [![Live Demo](https://img.shields.io/badge/Live%20Demo-Colab-orange?logo=googlecolab&logoColor=white)](https://colab.research.google.com/drive/11KqBQogdIjwC6UXAGVeD4cfq_VclUC_I?usp=sharing)
25
+ [![Dataset](https://img.shields.io/badge/Datasets-Hugging%20Face-yellow?logo=huggingface&logoColor=black)](https://huggingface.co/datasets/buddhi19/SyntheticGenV5)
 
 
26
 
27
  </div>
28
 
 
30
 
31
  ## ๐ŸŒŸ Overview
32
 
33
+ **SyntheticGen** is the official implementation for the paper [Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation](https://huggingface.co/papers/2602.04749). It tackles the long-tail distribution problem in remote-sensing datasets (specifically LoveDA) by generating synthetic imagery with *explicit control* over class ratios.
 
 
 
 
 
34
 
35
  ### โœจ Highlights
36
+ - **Two-stage pipeline**: Ratio-conditioned layout D3PM + ControlNet image synthesis.
37
+ - **Controllable Augmentation**: Specify exact proportions of each land cover class (e.g., `building:0.4`).
38
+ - **Data-Centric Strategy**: Improves segmentation performance by adding the *right* samples to the training set.
39
 
40
  <div align="center">
41
+ <img src="https://raw.githubusercontent.com/Buddhi19/SyntheticGen/main/docs/results.png" alt="SyntheticGen Results" width="100%">
42
  </div>
43
 
 
44
  ---
45
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  ## ๐Ÿš€ Quick Start
47
 
48
  ### Installation
 
51
  cd SyntheticGen
52
  pip install -r requirements.txt
53
  ```
 
 
 
 
 
 
 
 
 
 
54
 
55
  ### Generate Your First Synthetic Image
56
+ To generate a synthetic image-label pair using a specific configuration:
57
  ```bash
58
  python src/scripts/sample_pair.py \
59
  --config configs/sample_pair_ckpt40000_building0.4.yaml
 
63
 
64
  ## ๐Ÿ“š Usage
65
 
66
+ ### Training Pipeline
67
 
68
  **Stage A: Train Layout Generator (D3PM)**
69
  ```bash
 
71
  --config configs/train_layout_d3pm_masked_sparse_80k.yaml
72
  ```
73
 
 
 
 
 
 
 
74
  **Stage B: Train Image Generator (ControlNet)**
75
  ```bash
76
  python src/scripts/train_controlnet_ratio.py \
77
  --config configs/train_controlnet_ratio_loveda_1024.yaml
78
  ```
79
 
80
+ ### Inference / Sampling
81
+ **Override config parameters via CLI:**
 
 
 
 
 
 
 
82
  ```bash
83
  python src/scripts/sample_pair.py \
84
  --config configs/sample_pair_ckpt40000_building0.4.yaml \
 
88
 
89
  ---
90
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
  ## ๐Ÿ“„ Citation
92
  ```bibtex
93
  @misc{wijenayake2026mitigating,
 
103
 
104
  ---
105
 
 
 
 
 
 
106
  ## ๐Ÿ™ Acknowledgments
107
+ - LoveDA dataset creators for high-quality annotated remote sensing data.
108
+ - Hugging Face Diffusers for diffusion model infrastructure.
109
+ - ControlNet authors for controllable generation.