JeonghyeokDo commited on
Commit
bf48bd4
·
verified ·
1 Parent(s): 6aedfc0

Upload folder using huggingface_hub

Browse files
Files changed (50) hide show
  1. README.md +24 -4
  2. baselines/README.md +313 -0
  3. baselines/licenses/LICENSE-SoftPool-MIT.txt +21 -0
  4. baselines/licenses/LICENSE-bbdm.txt +21 -0
  5. baselines/licenses/LICENSE-cbbdm.txt +21 -0
  6. baselines/licenses/LICENSE-cdiffset.txt +21 -0
  7. baselines/licenses/LICENSE-cyclegan.txt +58 -0
  8. baselines/licenses/LICENSE-hidiff.txt +201 -0
  9. baselines/licenses/LICENSE-p2phd.txt +45 -0
  10. baselines/licenses/LICENSE-pix2pix.txt +58 -0
  11. baselines/licenses/LICENSE-resshift.txt +35 -0
  12. baselines/licenses/LICENSE-sd21ft.txt +21 -0
  13. baselines/licenses/LICENSE-spade.txt +177 -0
  14. baselines/licenses/NO-UPSTREAM-LICENSE.md +189 -0
  15. baselines/qxs-saropt/bbdm/README.md +136 -0
  16. baselines/qxs-saropt/cbbdm/README.md +153 -0
  17. baselines/qxs-saropt/cdiffset/README.md +164 -0
  18. baselines/qxs-saropt/cdiffset/config.json +45 -0
  19. baselines/qxs-saropt/conddiff/README.md +173 -0
  20. baselines/qxs-saropt/controlnet/README.md +127 -0
  21. baselines/qxs-saropt/controlnet/config.json +56 -0
  22. baselines/qxs-saropt/cyclegan/README.md +169 -0
  23. baselines/qxs-saropt/ddpm/README.md +174 -0
  24. baselines/qxs-saropt/e3diff/README.md +177 -0
  25. baselines/qxs-saropt/hidiff/README.md +162 -0
  26. baselines/qxs-saropt/p2phd/README.md +142 -0
  27. baselines/qxs-saropt/pix2pix/README.md +141 -0
  28. baselines/qxs-saropt/resshift/README.md +144 -0
  29. baselines/qxs-saropt/sd21ft/README.md +156 -0
  30. baselines/qxs-saropt/sd21ft/config.json +45 -0
  31. baselines/qxs-saropt/spade/README.md +176 -0
  32. baselines/qxs-saropt/stegogan/README.md +200 -0
  33. baselines/sar2opt/bbdm/README.md +138 -0
  34. baselines/sar2opt/cbbdm/README.md +155 -0
  35. baselines/sar2opt/cdiffset/README.md +166 -0
  36. baselines/sar2opt/cdiffset/config.json +45 -0
  37. baselines/sar2opt/conddiff/README.md +175 -0
  38. baselines/sar2opt/controlnet/README.md +128 -0
  39. baselines/sar2opt/controlnet/config.json +56 -0
  40. baselines/sar2opt/cyclegan/README.md +172 -0
  41. baselines/sar2opt/ddpm/README.md +176 -0
  42. baselines/sar2opt/e3diff/README.md +179 -0
  43. baselines/sar2opt/hidiff/README.md +164 -0
  44. baselines/sar2opt/p2phd/README.md +145 -0
  45. baselines/sar2opt/pix2pix/README.md +144 -0
  46. baselines/sar2opt/resshift/README.md +146 -0
  47. baselines/sar2opt/sd21ft/README.md +158 -0
  48. baselines/sar2opt/sd21ft/config.json +45 -0
  49. baselines/sar2opt/spade/README.md +179 -0
  50. baselines/sar2opt/stegogan/README.md +205 -0
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
- license: cc-by-nc-4.0
3
- license_name: cc-by-nc-4.0
4
  pipeline_tag: image-to-image
5
  library_name: diffusers
6
  tags:
@@ -20,7 +20,7 @@ from scratch inside a frozen high-fidelity autoencoder latent space.
20
  - **Paper:** https://arxiv.org/abs/{{ARXIV_ID}}
21
  - **Code:** https://github.com/KAIST-VICLab/ReFlowSET
22
  - **Project page:** https://kaist-viclab.github.io/ReFlowSET_site/
23
- - **Comparison-method weights:** [`JeonghyeokDo/ReFlowSET-baselines`](https://huggingface.co/JeonghyeokDo/ReFlowSET-baselines)
24
 
25
  ## Checkpoints
26
 
@@ -115,9 +115,29 @@ QXS-SAROPT and SAR2Opt only, one dataset per arm, trained from scratch. No
115
  pretraining corpus. Neither dataset is redistributed. QXS-SAROPT requires citing
116
  arXiv:2103.08259 for research use.
117
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118
  ## Licence and provenance
119
 
120
- **Weights: CC BY-NC 4.0. Code: Apache-2.0.**
 
 
 
 
 
 
121
 
122
  The frozen autoencoder bundled in `vae/` is the **Apache-2.0** autoencoder from
123
  [`black-forest-labs/FLUX.2-klein-base-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B),
 
1
  ---
2
+ license: other
3
+ license_name: mixed-see-model-card
4
  pipeline_tag: image-to-image
5
  library_name: diffusers
6
  tags:
 
20
  - **Paper:** https://arxiv.org/abs/{{ARXIV_ID}}
21
  - **Code:** https://github.com/KAIST-VICLab/ReFlowSET
22
  - **Project page:** https://kaist-viclab.github.io/ReFlowSET_site/
23
+ - **Comparison-method weights:** [`baselines/`](https://huggingface.co/JeonghyeokDo/ReFlowSET/tree/main/baselines) in this repository — all fifteen prior methods, both datasets
24
 
25
  ## Checkpoints
26
 
 
115
  pretraining corpus. Neither dataset is redistributed. QXS-SAROPT requires citing
116
  arXiv:2103.08259 for research use.
117
 
118
+ ## Comparison methods, in this same repository
119
+
120
+ `baselines/` holds the **fifteen prior methods of the paper's main table**,
121
+ retrained by us on the same splits and scored by the same evaluator — thirty
122
+ checkpoints, both datasets, each with its own card, its measured row, its
123
+ upstream repository and its licence. Start at
124
+ [`baselines/README.md`](https://huggingface.co/JeonghyeokDo/ReFlowSET/blob/main/baselines/README.md).
125
+
126
+ ```python
127
+ from diffusers import UNet2DConditionModel
128
+ unet = UNet2DConditionModel.from_pretrained(
129
+ "JeonghyeokDo/ReFlowSET", subfolder="baselines/qxs-saropt/cdiffset")
130
+ ```
131
+
132
  ## Licence and provenance
133
 
134
+ **This repository is mixed-licence, so the Hub tag is `other`.** ReFlowSET's own
135
+ weights are **CC BY-NC 4.0** and the code is **Apache-2.0**. Everything under
136
+ `baselines/` is a third-party method carrying its own terms — MIT, BSD-3-Clause,
137
+ Apache-2.0, CreativeML-OpenRAIL-M, CC BY-NC-SA 4.0, S-Lab 1.0, and four with no
138
+ upstream licence file at all. The per-method cards state each one, and the texts
139
+ that must travel are in `baselines/licenses/`. Check the method you intend to
140
+ use; the repo-level tag is not a substitute.
141
 
142
  The frozen autoencoder bundled in `vae/` is the **Apache-2.0** autoencoder from
143
  [`black-forest-labs/FLUX.2-klein-base-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B),
baselines/README.md ADDED
@@ -0,0 +1,313 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: per-method-see-below
4
+ library_name: pytorch
5
+ pipeline_tag: image-to-image
6
+ tags:
7
+ - sar-to-eo
8
+ - image-translation
9
+ - remote-sensing
10
+ - baselines
11
+ - benchmark
12
+ - reflowset
13
+ ---
14
+
15
+ # ReFlowSET — retrained comparison-method checkpoints
16
+
17
+ The fifteen comparison methods of the **ReFlowSET** SAR-to-EO benchmark, on both
18
+ of its datasets: **QXS-SAROPT** (256 px) and **SAR2Opt** (512 px). Thirty
19
+ checkpoint cells in total.
20
+
21
+ **Every checkpoint here was retrained by us.** None is an original author's
22
+ released checkpoint. Each method was trained on the same official split, at the
23
+ same resolution, under a budget stated in optimizer updates, and every cell was
24
+ then scored through one evaluation pipeline with an identity-collapse and
25
+ ground-truth-leak audit. That is what makes the rows below comparable **to each
26
+ other** — and, for exactly the same reason, not comparable with the numbers in
27
+ the methods' own papers.
28
+
29
+ ReFlowSET itself is not in this repository. It lives at
30
+ [JeonghyeokDo/ReFlowSET](https://huggingface.co/JeonghyeokDo/ReFlowSET); code at https://github.com/KAIST-VICLab/ReFlowSET; project page
31
+ at https://kaist-viclab.github.io/ReFlowSET_site/.
32
+
33
+ ## Results — all sixteen rows, both datasets
34
+
35
+ Our own measurements, on the images each checkpoint produced. No number below
36
+ is copied from any paper.
37
+
38
+ ### QXS-SAROPT — n = 3,999, 256 px
39
+
40
+ | Method | Venue | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
41
+ |---|---|---|---|---|---|---|
42
+ | *General image-to-image* | | | | | | |
43
+ | pix2pix | CVPR'17 | 174.6 | 0.373 | 0.665 | 0.203 | 12.33 |
44
+ | CycleGAN ⚠ | ICCV'17 | 104.4 | 0.376 | 0.653 | 0.262 | 12.92 |
45
+ | pix2pixHD | CVPR'18 | 85.7 | 0.298 | 0.573 | 0.358 | 16.13 |
46
+ | SPADE | CVPR'19 | 90.7 | 0.292 | 0.599 | 0.320 | 14.53 |
47
+ | DDPM (SR3-class) | TPAMI'22 | 43.8 | 0.311 | 0.620 | 0.359 | 14.04 |
48
+ | SD2.1 fine-tune only | CVPR'22 | 19.1 | 0.257 | 0.561 | 0.348 | 15.40 |
49
+ | BBDM | CVPR'23 | 76.6 | 0.270 | 0.568 | 0.352 | 15.34 |
50
+ | ControlNet | ICCV'23 | 50.4 | 0.307 | 0.604 | 0.297 | 13.42 |
51
+ | HI-Diff | NeurIPS'23 | 324.3 | 0.539 | 0.692 | 0.457 | 17.10 |
52
+ | ResShift | NeurIPS'23 | 140.2 | 0.334 | 0.607 | 0.217 | 14.20 |
53
+ | StegoGAN ⚠ | CVPR'24 | 106.8 | 0.384 | 0.658 | 0.254 | 12.96 |
54
+ | *SAR-to-EO* | | | | | | |
55
+ | Conditional Diffusion | GRSL'24 | 88.6 | 0.355 | 0.730 | 0.213 | 11.55 |
56
+ | cBBDM | GRSL'25 | 50.6 | 0.246 | 0.539 | 0.372 | 16.02 |
57
+ | E3Diff | GRSL'25 | 47.8 | 0.278 | 0.530 | 0.302 | 16.44 |
58
+ | C-DiffSET | TCSVT'26 | 19.9 | 0.233 | 0.526 | 0.380 | 16.92 |
59
+ | *Ours* | | | | | | |
60
+ | **ReFlowSET (ours)** | — | 19.1 | 0.231 | 0.534 | 0.355 | 16.09 |
61
+
62
+ ### SAR2Opt — n = 627, 512 px (centre crops of the 600 px tiles)
63
+
64
+ | Method | Venue | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
65
+ |---|---|---|---|---|---|---|
66
+ | *General image-to-image* | | | | | | |
67
+ | pix2pix | CVPR'17 | 261.9 | 0.347 | 0.657 | 0.199 | 13.39 |
68
+ | CycleGAN ⚠ | ICCV'17 | 143.5 | 0.330 | 0.650 | 0.178 | 12.90 |
69
+ | pix2pixHD | CVPR'18 | 146.3 | 0.283 | 0.567 | 0.268 | 15.95 |
70
+ | SPADE | CVPR'19 | 142.5 | 0.265 | 0.597 | 0.234 | 14.47 |
71
+ | DDPM (SR3-class) | TPAMI'22 | 122.5 | 0.295 | 0.610 | 0.313 | 13.65 |
72
+ | SD2.1 fine-tune only | CVPR'22 | 71.8 | 0.211 † | 0.541 | 0.293 | 16.24 |
73
+ | BBDM | CVPR'23 | 143.1 | 0.290 | 0.590 | 0.276 | 15.29 |
74
+ | ControlNet | ICCV'23 | 140.5 | 0.350 | 0.643 | 0.217 | 11.73 |
75
+ | HI-Diff | NeurIPS'23 | 319.8 | 0.473 | 0.692 | 0.384 | 17.36 |
76
+ | ResShift | NeurIPS'23 | 141.7 | 0.304 | 0.597 | 0.177 | 14.31 |
77
+ | StegoGAN ⚠ | CVPR'24 | 150.1 | 0.347 | 0.655 | 0.158 | 12.47 |
78
+ | *SAR-to-EO* | | | | | | |
79
+ | Conditional Diffusion | GRSL'24 | 211.8 | 0.415 | 0.686 | 0.248 | 12.48 |
80
+ | cBBDM | GRSL'25 | 222.3 | 0.377 | 0.571 | 0.361 | 17.05 |
81
+ | E3Diff | GRSL'25 | 104.7 | 0.232 | 0.529 | 0.249 | 16.09 |
82
+ | C-DiffSET | TCSVT'26 | 78.1 | 0.214 † | 0.529 | 0.314 | 16.81 |
83
+ | *Ours* | | | | | | |
84
+ | **ReFlowSET (ours)** | — | 66.3 | 0.185 | 0.522 | 0.287 | 16.06 |
85
+
86
+ ⚠ **fails the input-copy audit** — see *Identity collapse* below.
87
+ † **these two DISTS values were re-measured.** The image dumps for these two
88
+ SAR2Opt cells were regenerated after their last extended-metrics pass, and that
89
+ pass's resume check keys on the number of images rather than on their content, so
90
+ it skipped them and their stored DISTS went stale. Both were re-measured on the
91
+ current dumps at full n=627 and the table prints the re-measurement (SD2.1-FT
92
+ 0.212 -> **0.211**; C-DiffSET 0.2141 -> **0.2136**, unchanged at three decimals).
93
+ The harness was validated on a cell that is *not* stale, whose DISTS it
94
+ reproduces to 6e-6. FID, LPIPS, SSIM and PSNR come from a pass that rescores
95
+ every cell on every run and were never affected.
96
+
97
+ ## Read before you compare these numbers with a paper
98
+
99
+ **These numbers are not comparable with the source papers, and are not meant to
100
+ be.** Three reasons, all deliberate:
101
+
102
+ 1. **Budgets are matched across the benchmark, not to each method's own
103
+ recipe.** Several methods are trained here at a fraction of their published
104
+ schedule — HI-Diff, for instance, is trained for 50,000 updates against an
105
+ upstream recipe of 600,000, with progressive resizing disabled. Where we
106
+ departed from a published recipe, the method's own card says so and says by
107
+ how much.
108
+ 2. **Budgets are quoted in optimizer updates, never epochs.** The two datasets
109
+ differ by more than tenfold in size (16,001 versus 1,450 training pairs), so
110
+ an epoch count means something completely different on each. Comparing
111
+ unpaired methods by epochs produced two successive wrong conclusions in our
112
+ own work before we changed the unit.
113
+ 3. **One evaluation protocol for everyone.** Same test items, same resolution,
114
+ same metric implementations, same ground truth. SAR2Opt is **centre-cropped**
115
+ to 512 px and never resized, in training and in evaluation alike.
116
+
117
+ ### The LPIPS convention
118
+
119
+ LPIPS here is **LPIPS-VGG on inputs mapped to `[-1, 1]`** before the network —
120
+ `lpips_vgg(x * 2 - 1, y * 2 - 1)` with the library's default `normalize=False`.
121
+ The other convention, which several released evaluators use, hands the network
122
+ `[0, 1]` inputs *unscaled* (still `normalize=False`) and produces a
123
+ **systematically lower number** — about 0.05 lower — on the same images. Note
124
+ that `[0, 1]` inputs with `normalize=True` is the *same* thing as ours, not the
125
+ other one; only the flag differs. Both are called "LPIPS" in the literature. If you compare a number from
126
+ this table against one from a paper, check which convention that paper used
127
+ first; otherwise the comparison is meaningless. The same care applies in reverse
128
+ to anyone quoting these numbers.
129
+
130
+ PSNR and SSIM are per-image torchmetrics with `data_range=1`. FID is
131
+ `pytorch-fid` against the size-matched ground-truth test set. DISTS is the
132
+ standard implementation on the same pairs. Where the generated size and the
133
+ ground-truth size differ, the ground truth is centre-cropped to match.
134
+
135
+ ### Identity collapse
136
+
137
+ **Four of the thirty cells fail an input-copy audit**: CycleGAN and StegoGAN, on
138
+ both datasets. Their outputs are closer to the SAR image they were given than to
139
+ the EO image they were asked for — mean |output − SAR| < mean |output − ground
140
+ truth| — which is the signature of a model that has learned to pass its input
141
+ through rather than translate it.
142
+
143
+ | cell | ratio (SAR distance ÷ GT distance) | verdict |
144
+ |---|---|---|
145
+ | QXS-SAROPT CycleGAN | 0.847 | COLLAPSED |
146
+ | QXS-SAROPT StegoGAN | 0.919 | COLLAPSED |
147
+ | SAR2Opt CycleGAN | 0.795 | COLLAPSED |
148
+ | SAR2Opt StegoGAN | 0.758 | COLLAPSED |
149
+
150
+ **Their metric values are reported unchanged and marked in place.** They are what
151
+ the released implementation does at its own published protocol, and a benchmark
152
+ that quietly substitutes a better-behaved variant is not reporting the method.
153
+ But they should not be read as translation quality. No other cell on either
154
+ dataset collapses — in particular DDPM, Conditional Diffusion and BBDM do **not**
155
+ collapse here, whatever they may do elsewhere.
156
+
157
+ The companion leak audit is clean: no cell on either dataset shows a generation
158
+ that matches its own ground truth anomalously better than it matches other
159
+ images' ground truth. Per-cell numbers are on each method's card.
160
+
161
+ **One oracle you must not use.** StegoGAN's `fake_B` output is computed as
162
+ `netG_A(real_A, latent(real_B))` — conditioned on the ground truth, and
163
+ impossible to produce at deployment. Only `fake_B_clean = netG_A(real_A)` is a
164
+ prediction, and it is what the StegoGAN rows above report. Any published StegoGAN
165
+ number should say which of the two it is.
166
+
167
+ ## Layout
168
+
169
+ ```
170
+ <dataset>/<method>/<weight files> + README.md
171
+ dataset ∈ { qxs-saropt, sar2opt }
172
+ licenses/ third-party licence texts, one per method
173
+ ```
174
+
175
+ Every one of the thirty folders carries its own `README.md` with that cell's
176
+ architecture, the file-by-file contents, the training budget in optimizer
177
+ updates, the measured metrics, a copy-pasteable load-and-translate snippet, the
178
+ traps specific to that method, and its licence stated factually.
179
+
180
+ | method | folder | files | upstream code |
181
+ |---|---|---|---|
182
+ | pix2pix | `pix2pix/` | `net_G.pth` | `junyanz/pytorch-CycleGAN-and-pix2pix` |
183
+ | CycleGAN | `cyclegan/` | `net_G_A.pth` (**SAR→EO**), `net_G_B.pth` | `junyanz/pytorch-CycleGAN-and-pix2pix` |
184
+ | pix2pixHD | `p2phd/` | `net_G.pth` | `NVIDIA/pix2pixHD` |
185
+ | SPADE | `spade/` | `net_G.pth` | `NVlabs/SPADE` |
186
+ | DDPM (SR3-class) | `ddpm/` | `gen.pth` | `DeepSARRS/E3Diff`, stage 1 |
187
+ | SD2.1 fine-tune only | `sd21ft/` | `diffusion_pytorch_model.safetensors`, `config.json` | `KAIST-VICLab/C-DiffSET`, stage 1, on SD 2.1-base |
188
+ | BBDM | `bbdm/` | `last_model.pth` | `xuekt98/BBDM` |
189
+ | ControlNet | `controlnet/` | `diffusion_pytorch_model.safetensors`, `config.json` | `huggingface/diffusers` `train_controlnet.py`, on SD 2.1-base |
190
+ | HI-Diff | `hidiff/` | `S1_*.pth` ×2, `S2_*.pth` ×3 (**only the three `S2_` files are needed at inference**) | `zhengchen1999/HI-Diff` |
191
+ | ResShift | `resshift/` | `ema_model.pth` | `zsyOAOA/ResShift` |
192
+ | StegoGAN | `stegogan/` | `net_G_A.pth` (**SAR→EO**), `net_G_B.pth` | `sian-wusidi/StegoGAN` |
193
+ | Conditional Diffusion | `conddiff/` | `ema_final.pt` | `Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation` |
194
+ | cBBDM | `cbbdm/` | `last_model.pth` | `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical` |
195
+ | E3Diff | `e3diff/` | `gen.pth` | `DeepSARRS/E3Diff`, stage 2 |
196
+ | C-DiffSET | `cdiffset/` | `diffusion_pytorch_model.safetensors`, `config.json` | `KAIST-VICLab/C-DiffSET`, stage 2 |
197
+
198
+ ### Three checkpoints these weights need and this repository does not carry
199
+
200
+ * **BBDM and cBBDM** need the CompVis latent-diffusion **vq-f4 VQGAN**
201
+ (`model.ckpt`, 756,175,527 bytes) at the path named in their config — at
202
+ construction time, even though the same weights are also inside
203
+ `last_model.pth`. MIT.
204
+ * **ResShift** needs the CompVis **vq-f4 autoencoder** (221,364,711 bytes). MIT.
205
+ * **SD2.1-FT, ControlNet and C-DiffSET** need Stable Diffusion 2.1-base for the
206
+ frozen VAE, text encoder, tokenizer and scheduler. Their cards name the exact
207
+ repository we loaded.
208
+ * **DDPM and E3Diff** need no extra checkpoint, but do need **PPB-despeckled and
209
+ Canny condition images** computed from each SAR input; neither can be run from
210
+ a SAR file alone.
211
+
212
+ ## Licences
213
+
214
+ **A single repository-wide licence tag would be wrong for this collection.** The
215
+ fifteen methods fall into three tiers, and the restrictive terms in tiers B and C
216
+ travel with the individual cells. Each method's card states its position in full;
217
+ the licence texts are in `licenses/`.
218
+
219
+ | method | upstream licence | redistribution | what propagates |
220
+ |---|---|---|---|
221
+ | pix2pix | multi-part BSD (CycleGAN / pix2pix / DCGAN notices) | ✅ with the notice | no-endorsement (DCGAN part) |
222
+ | CycleGAN | same file as pix2pix | ✅ with the notice | same |
223
+ | pix2pixHD | BSD (NVIDIA 2019) + bundled CycleGAN notice | ✅ with both notices | none |
224
+ | **SPADE** | **CC BY-NC-SA 4.0** | ⚠️ **non-commercial only** | **NonCommercial + ShareAlike** — this cell is itself distributed under CC BY-NC-SA 4.0 |
225
+ | **DDPM (SR3-class)** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown; there is no licence to read |
226
+ | SD2.1 fine-tune only | MIT code (C-DiffSET) / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |
227
+ | BBDM | MIT | ✅ | none (needs the external vq-f4 VQGAN, MIT) |
228
+ | ControlNet | Apache-2.0 code (diffusers) / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |
229
+ | HI-Diff | Apache-2.0 | ✅ | ship the licence, keep notices, state your modifications |
230
+ | **ResShift** | **S-Lab License 1.0** | ⚠️ **non-commercial only** | **NonCommercial**; no-endorsement |
231
+ | **StegoGAN** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown |
232
+ | **Conditional Diffusion** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown (the guided-diffusion parts it forks are MIT) |
233
+ | cBBDM | MIT | ✅ | none (needs the external vq-f4 VQGAN, MIT) |
234
+ | **E3Diff** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown |
235
+ | C-DiffSET | MIT code / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |
236
+
237
+ ### The four with no upstream licence
238
+
239
+ **StegoGAN, E3Diff, DDPM (SR3-class) and Conditional Diffusion are derived from
240
+ upstream code that publishes no licence at all** — no LICENSE, LICENCE, COPYING
241
+ or NOTICE file, no licence section in the README, and no licence declared to
242
+ GitHub. Under default copyright, all rights remain reserved by those authors and
243
+ **no express permission to redistribute derived work has been granted** to us or
244
+ to you.
245
+
246
+ We publish these four anyway, so that the benchmark is reproducible end to end,
247
+ and we say so plainly rather than implying a permission that does not exist. If
248
+ you intend to redistribute them or build on them, assess that for yourself, and
249
+ consider asking the upstream authors directly.
250
+ `licenses/NO-UPSTREAM-LICENSE.md` records, per method, exactly what we checked
251
+ and exactly what we found.
252
+
253
+ ### Files in `licenses/`
254
+
255
+ `LICENSE-pix2pix.txt`, `LICENSE-cyclegan.txt`, `LICENSE-p2phd.txt`,
256
+ `LICENSE-spade.txt`, `LICENSE-bbdm.txt`, `LICENSE-cbbdm.txt`,
257
+ `LICENSE-hidiff.txt`, `LICENSE-resshift.txt`, `LICENSE-cdiffset.txt`,
258
+ `LICENSE-sd21ft.txt` — one per method, copied verbatim from the upstream
259
+ repository we trained with. `LICENSE-SoftPool-MIT.txt` is a vendored dependency
260
+ of the E3Diff code base and is included because it is genuinely part of what
261
+ those two cells were trained with; it is **not** a licence for E3Diff.
262
+ `NO-UPSTREAM-LICENSE.md` covers the four methods with no grant.
263
+ `LICENSE-WEIGHTS.md` at the repository root is the CreativeML Open RAIL++-M
264
+ text, including Attachment A, which governs the three Stable-Diffusion-derived
265
+ cells.
266
+
267
+ ## Datasets
268
+
269
+ Neither corpus is redistributed here, in any form, and neither may be
270
+ redistributed from here.
271
+
272
+ * **QXS-SAROPT** publishes no licence file; its one stated term is a citation
273
+ requirement — *"This paper must be cited when the dataset is used for research
274
+ purposes"* (Huang et al., [arXiv:2103.08259](https://arxiv.org/abs/2103.08259)).
275
+ Its distribution is request-gated by the authors. Obtain it from them.
276
+ * **SAR2Opt** ships an MIT LICENSE, `Copyright (c) 2021 MarsZhaoYT`. The MIT
277
+ text grants rights in "the Software"; the repository does not name the imagery
278
+ sources, so that file is best read as covering the authors' packaging and code.
279
+ Citation requested: Zhao, Celik, Liu and Li, *A Comparative Analysis of
280
+ GAN-based Methods for SAR-to-Optical Image Translation*, IEEE GRSL, 2022.
281
+ * **The optical side of both datasets is Google-Earth-derived.** Copyright in
282
+ that imagery is held by third parties, not by the dataset authors and not by
283
+ us. No such imagery appears in this repository.
284
+
285
+ ## A note on two venue labels
286
+
287
+ The table above prints the venue strings as they appear in the ReFlowSET paper.
288
+ Two of them do not match the year in the upstream repositories' own citation
289
+ blocks, and both refer to the same paper under an early-access versus issue year:
290
+
291
+ * **Conditional Diffusion** — the authors' bibtex says **2023**,
292
+ [doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143).
293
+ * **E3Diff** — the authors' bibtex says **2024**,
294
+ [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).
295
+
296
+ Cite by DOI.
297
+
298
+ ## Citation
299
+
300
+ If you use this benchmark or any of these weights, please cite ReFlowSET:
301
+
302
+ ```bibtex
303
+ @article{do2026reflowset,
304
+ title = {ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation},
305
+ author = {Jeonghyeok Do<sup>1</sup>, Seungchul Lee<sup>2</sup>, Munchurl Kim<sup>1*</sup>},
306
+ journal = {arXiv preprint arXiv:{{ARXIV_ID}}},
307
+ year = {2026}
308
+ }
309
+ ```
310
+
311
+ **And please cite each method's original paper** — every model card carries the
312
+ reference — **and both dataset papers.** The QXS-SAROPT citation is required by
313
+ that dataset's own terms.
baselines/licenses/LICENSE-SoftPool-MIT.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2020 Alexandros Stergiou
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
baselines/licenses/LICENSE-bbdm.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2023 xuekt98
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
baselines/licenses/LICENSE-cbbdm.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2025 egshkim
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
baselines/licenses/LICENSE-cdiffset.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 KAIST VICLab
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
baselines/licenses/LICENSE-cyclegan.txt ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright (c) 2017, Jun-Yan Zhu and Taesung Park
2
+ All rights reserved.
3
+
4
+ Redistribution and use in source and binary forms, with or without
5
+ modification, are permitted provided that the following conditions are met:
6
+
7
+ * Redistributions of source code must retain the above copyright notice, this
8
+ list of conditions and the following disclaimer.
9
+
10
+ * Redistributions in binary form must reproduce the above copyright notice,
11
+ this list of conditions and the following disclaimer in the documentation
12
+ and/or other materials provided with the distribution.
13
+
14
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
15
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
17
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
18
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
19
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
20
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
21
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
22
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
23
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
24
+
25
+
26
+ --------------------------- LICENSE FOR pix2pix --------------------------------
27
+ BSD License
28
+
29
+ For pix2pix software
30
+ Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu
31
+ All rights reserved.
32
+
33
+ Redistribution and use in source and binary forms, with or without
34
+ modification, are permitted provided that the following conditions are met:
35
+
36
+ * Redistributions of source code must retain the above copyright notice, this
37
+ list of conditions and the following disclaimer.
38
+
39
+ * Redistributions in binary form must reproduce the above copyright notice,
40
+ this list of conditions and the following disclaimer in the documentation
41
+ and/or other materials provided with the distribution.
42
+
43
+ ----------------------------- LICENSE FOR DCGAN --------------------------------
44
+ BSD License
45
+
46
+ For dcgan.torch software
47
+
48
+ Copyright (c) 2015, Facebook, Inc. All rights reserved.
49
+
50
+ Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
51
+
52
+ Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
53
+
54
+ Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
55
+
56
+ Neither the name Facebook nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
57
+
58
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
baselines/licenses/LICENSE-hidiff.txt ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright 2023 HI-Diff Authors
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
baselines/licenses/LICENSE-p2phd.txt ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright (C) 2019 NVIDIA Corporation. Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu.
2
+ BSD License. All rights reserved.
3
+
4
+ Redistribution and use in source and binary forms, with or without
5
+ modification, are permitted provided that the following conditions are met:
6
+
7
+ * Redistributions of source code must retain the above copyright notice, this
8
+ list of conditions and the following disclaimer.
9
+
10
+ * Redistributions in binary form must reproduce the above copyright notice,
11
+ this list of conditions and the following disclaimer in the documentation
12
+ and/or other materials provided with the distribution.
13
+
14
+ THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL
15
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ANY PARTICULAR PURPOSE.
16
+ IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL
17
+ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
18
+ WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
19
+ OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
20
+
21
+
22
+ --------------------------- LICENSE FOR pytorch-CycleGAN-and-pix2pix ----------------
23
+ Copyright (c) 2017, Jun-Yan Zhu and Taesung Park
24
+ All rights reserved.
25
+
26
+ Redistribution and use in source and binary forms, with or without
27
+ modification, are permitted provided that the following conditions are met:
28
+
29
+ * Redistributions of source code must retain the above copyright notice, this
30
+ list of conditions and the following disclaimer.
31
+
32
+ * Redistributions in binary form must reproduce the above copyright notice,
33
+ this list of conditions and the following disclaimer in the documentation
34
+ and/or other materials provided with the distribution.
35
+
36
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
37
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
38
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
39
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
40
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
41
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
42
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
43
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
44
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
45
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
baselines/licenses/LICENSE-pix2pix.txt ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright (c) 2017, Jun-Yan Zhu and Taesung Park
2
+ All rights reserved.
3
+
4
+ Redistribution and use in source and binary forms, with or without
5
+ modification, are permitted provided that the following conditions are met:
6
+
7
+ * Redistributions of source code must retain the above copyright notice, this
8
+ list of conditions and the following disclaimer.
9
+
10
+ * Redistributions in binary form must reproduce the above copyright notice,
11
+ this list of conditions and the following disclaimer in the documentation
12
+ and/or other materials provided with the distribution.
13
+
14
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
15
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
17
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
18
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
19
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
20
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
21
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
22
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
23
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
24
+
25
+
26
+ --------------------------- LICENSE FOR pix2pix --------------------------------
27
+ BSD License
28
+
29
+ For pix2pix software
30
+ Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu
31
+ All rights reserved.
32
+
33
+ Redistribution and use in source and binary forms, with or without
34
+ modification, are permitted provided that the following conditions are met:
35
+
36
+ * Redistributions of source code must retain the above copyright notice, this
37
+ list of conditions and the following disclaimer.
38
+
39
+ * Redistributions in binary form must reproduce the above copyright notice,
40
+ this list of conditions and the following disclaimer in the documentation
41
+ and/or other materials provided with the distribution.
42
+
43
+ ----------------------------- LICENSE FOR DCGAN --------------------------------
44
+ BSD License
45
+
46
+ For dcgan.torch software
47
+
48
+ Copyright (c) 2015, Facebook, Inc. All rights reserved.
49
+
50
+ Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
51
+
52
+ Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
53
+
54
+ Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
55
+
56
+ Neither the name Facebook nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
57
+
58
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
baselines/licenses/LICENSE-resshift.txt ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ S-Lab License 1.0
2
+
3
+ Copyright 2022 S-Lab
4
+
5
+ Redistribution and use for non-commercial purpose in source and
6
+ binary forms, with or without modification, are permitted provided
7
+ that the following conditions are met:
8
+
9
+ 1. Redistributions of source code must retain the above copyright
10
+ notice, this list of conditions and the following disclaimer.
11
+
12
+ 2. Redistributions in binary form must reproduce the above copyright
13
+ notice, this list of conditions and the following disclaimer in
14
+ the documentation and/or other materials provided with the
15
+ distribution.
16
+
17
+ 3. Neither the name of the copyright holder nor the names of its
18
+ contributors may be used to endorse or promote products derived
19
+ from this software without specific prior written permission.
20
+
21
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
22
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
23
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
24
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
25
+ HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
26
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
27
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
28
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
29
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
30
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
31
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
32
+
33
+ In the event that redistribution and/or use for commercial purpose in
34
+ source or binary forms, with or without modification is required,
35
+ please contact the contributor(s) of the work.
baselines/licenses/LICENSE-sd21ft.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 KAIST VICLab
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
baselines/licenses/LICENSE-spade.txt ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## creative commons
2
+
3
+ # Attribution-NonCommercial-ShareAlike 4.0 International
4
+
5
+ Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible.
6
+
7
+ ### Using Creative Commons Public Licenses
8
+
9
+ Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses.
10
+
11
+ * __Considerations for licensors:__ Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC-licensed material, or material used under an exception or limitation to copyright. [More considerations for licensors](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensors).
12
+
13
+ * __Considerations for the public:__ By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. [More considerations for the public](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensees).
14
+
15
+ ## Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License
16
+
17
+ By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
18
+
19
+ ### Section 1 – Definitions.
20
+
21
+ a. __Adapted Material__ means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
22
+
23
+ b. __Adapter's License__ means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
24
+
25
+ c. __BY-NC-SA Compatible License__ means a license listed at [creativecommons.org/compatiblelicenses](http://creativecommons.org/compatiblelicenses), approved by Creative Commons as essentially the equivalent of this Public License.
26
+
27
+ d. __Copyright and Similar Rights__ means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
28
+
29
+ e. __Effective Technological Measures__ means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
30
+
31
+ f. __Exceptions and Limitations__ means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
32
+
33
+ g. __License Elements__ means the license attributes listed in the name of a Creative Commons Public License. The License Elements of this Public License are Attribution, NonCommercial, and ShareAlike.
34
+
35
+ h. __Licensed Material__ means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
36
+
37
+ i. __Licensed Rights__ means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
38
+
39
+ h. __Licensor__ means the individual(s) or entity(ies) granting rights under this Public License.
40
+
41
+ i. __NonCommercial__ means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.
42
+
43
+ j. __Share__ means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
44
+
45
+ k. __Sui Generis Database Rights__ means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
46
+
47
+ l. __You__ means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
48
+
49
+ ### Section 2 – Scope.
50
+
51
+ a. ___License grant.___
52
+
53
+ 1. Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
54
+
55
+ A. reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and
56
+
57
+ B. produce, reproduce, and Share Adapted Material for NonCommercial purposes only.
58
+
59
+ 2. __Exceptions and Limitations.__ For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
60
+
61
+ 3. __Term.__ The term of this Public License is specified in Section 6(a).
62
+
63
+ 4. __Media and formats; technical modifications allowed.__ The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
64
+
65
+ 5. __Downstream recipients.__
66
+
67
+ A. __Offer from the Licensor – Licensed Material.__ Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
68
+
69
+ B. __Additional offer from the Licensor – Adapted Material.__ Every recipient of Adapted Material from You automatically receives an offer from the Licensor to exercise the Licensed Rights in the Adapted Material under the conditions of the Adapter’s License You apply.
70
+
71
+ C. __No downstream restrictions.__ You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
72
+
73
+ 6. __No endorsement.__ Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
74
+
75
+ b. ___Other rights.___
76
+
77
+ 1. Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
78
+
79
+ 2. Patent and trademark rights are not licensed under this Public License.
80
+
81
+ 3. To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes.
82
+
83
+ ### Section 3 – License Conditions.
84
+
85
+ Your exercise of the Licensed Rights is expressly made subject to the following conditions.
86
+
87
+ a. ___Attribution.___
88
+
89
+ 1. If You Share the Licensed Material (including in modified form), You must:
90
+
91
+ A. retain the following if it is supplied by the Licensor with the Licensed Material:
92
+
93
+ i. identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
94
+
95
+ ii. a copyright notice;
96
+
97
+ iii. a notice that refers to this Public License;
98
+
99
+ iv. a notice that refers to the disclaimer of warranties;
100
+
101
+ v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
102
+
103
+ B. indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
104
+
105
+ C. indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
106
+
107
+ 2. You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
108
+
109
+ 3. If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
110
+
111
+ b. ___ShareAlike.___
112
+
113
+ In addition to the conditions in Section 3(a), if You Share Adapted Material You produce, the following conditions also apply.
114
+
115
+ 1. The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-NC-SA Compatible License.
116
+
117
+ 2. You must include the text of, or the URI or hyperlink to, the Adapter's License You apply. You may satisfy this condition in any reasonable manner based on the medium, means, and context in which You Share Adapted Material.
118
+
119
+ 3. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, Adapted Material that restrict exercise of the rights granted under the Adapter's License You apply.
120
+
121
+ ### Section 4 – Sui Generis Database Rights.
122
+
123
+ Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
124
+
125
+ a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only;
126
+
127
+ b. if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material, including for purposes of Section 3(b); and
128
+
129
+ c. You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
130
+
131
+ For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
132
+
133
+ ### Section 5 – Disclaimer of Warranties and Limitation of Liability.
134
+
135
+ a. __Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.__
136
+
137
+ b. __To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.__
138
+
139
+ c. The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
140
+
141
+ ### Section 6 – Term and Termination.
142
+
143
+ a. This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
144
+
145
+ b. Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
146
+
147
+ 1. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
148
+
149
+ 2. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
150
+
151
+ For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
152
+
153
+ c. For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
154
+
155
+ d. Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
156
+
157
+ ### Section 7 – Other Terms and Conditions.
158
+
159
+ a. The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
160
+
161
+ b. Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
162
+
163
+ ### Section 8 – Interpretation.
164
+
165
+ a. For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
166
+
167
+ b. To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
168
+
169
+ c. No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
170
+
171
+ d. Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
172
+
173
+ ```
174
+ Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at [creativecommons.org/policies](http://creativecommons.org/policies), Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.
175
+
176
+ Creative Commons may be contacted at [creativecommons.org](http://creativecommons.org/).
177
+ ```
baselines/licenses/NO-UPSTREAM-LICENSE.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Four upstream code bases publish no licence
2
+
3
+ This file records, per method, **exactly what we checked and exactly what we
4
+ found**. It exists because four of the fifteen comparison methods in this
5
+ repository are derived from upstream code that grants no licence at all, and a
6
+ reader of these weights is entitled to that fact stated plainly rather than
7
+ inferred from a missing file.
8
+
9
+ The four are **StegoGAN**, **E3Diff**, **DDPM (SR3-class)** — which is the E3Diff
10
+ code base run in its stage-1 configuration — and **Conditional Diffusion
11
+ (CondDiff)**.
12
+
13
+ The checkpoints derived from them **are published here anyway**, as a deliberate
14
+ decision, so that the benchmark is reproducible end to end. We are not asserting
15
+ that a licence exists, and we are not granting one. Under default copyright, all
16
+ rights in the upstream code remain reserved by its authors, and **no express
17
+ permission to redistribute derived work has been granted to us or to you**. If
18
+ you intend to redistribute these four checkpoints or build on them, assess that
19
+ for yourself, and consider asking the upstream authors directly.
20
+
21
+ Everything below was checked on **2026-08-28**.
22
+
23
+ ---
24
+
25
+ ## What "checked" means
26
+
27
+ For each upstream we ran three tests. Where a test does not apply (for example,
28
+ a vendored tree that is not a git clone has no tracked-file list), that is stated
29
+ rather than skipped silently.
30
+
31
+ 1. **Filesystem.** `find <repo> -maxdepth 3 \( -iname '*licen*' -o -iname
32
+ 'COPYING*' -o -iname 'NOTICE*' \)` over the vendored working tree.
33
+ 2. **Version control.** `git ls-files | grep -iE 'licen|copying|notice'` at the
34
+ pinned commit, which catches a licence file that exists upstream but was
35
+ deleted locally.
36
+ 3. **Upstream host.** The GitHub REST API for the repository
37
+ (`license` field) and its `/license` endpoint.
38
+ 4. **README.** `grep -inE 'licen|copyright|all rights reserved|commercial'`
39
+ over the repository README, because some projects state terms in prose
40
+ instead of in a file.
41
+
42
+ ---
43
+
44
+ ## StegoGAN
45
+
46
+ *StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image
47
+ Translation*, Wu, Chen, Mermet, Hurni, Schindler, Gonthier and Landrieu,
48
+ CVPR 2024.
49
+
50
+ | test | result |
51
+ |---|---|
52
+ | upstream | `https://github.com/sian-wusidi/StegoGAN` (the official CVPR 2024 code) |
53
+ | pinned commit | `cad61997c0f82793444f60f81298142b80cdf3c1` |
54
+ | filesystem, maxdepth 3 | **no match** — no LICENSE, LICENCE, COPYING or NOTICE at any depth |
55
+ | tracked files at that commit | **0 of 52** tracked paths are licence-shaped |
56
+ | GitHub API | repository exists; `"license": null`; the `/license` endpoint returns **HTTP 404** |
57
+ | README | **no match** for `licen`, `copyright`, `all rights reserved` or `commercial` — the README has no licence section |
58
+
59
+ **Lineage.** StegoGAN's README states it is built on
60
+ `junyanz/pytorch-CycleGAN-and-pix2pix`. The parts of the tree that are
61
+ unmodified CycleGAN carry that project's BSD notice, reproduced here as
62
+ `LICENSE-cyclegan.txt`. **That notice does not extend to StegoGAN's own
63
+ contributions**, which are the mismatch-mask generator (`resnet_9blocks_maskv1` /
64
+ `maskv3`), the fusion block and the consistency/regularisation losses — i.e.
65
+ precisely the parts that make the released `net_G_A.pth` what it is.
66
+
67
+ **Conclusion: no licence. Weights published by decision, with this disclosure.**
68
+
69
+ ---
70
+
71
+ ## E3Diff
72
+
73
+ *Efficient End-to-End Diffusion Model for One-Step SAR-to-Optical Translation*,
74
+ Qin, Zou, Li and Zhang, IEEE GRSL, doi:10.1109/LGRS.2024.3506566.
75
+
76
+ | test | result |
77
+ |---|---|
78
+ | upstream | `https://github.com/DeepSARRS/E3Diff` |
79
+ | pinned commit | `38601093ab8f8e4b478144621f20890b100a3b74` |
80
+ | filesystem, maxdepth 3 | **one match, and it is not a grant for E3Diff**: `SoftPool/LICENSE.txt` |
81
+ | tracked files at that commit | **1 of 58** — the same `SoftPool/LICENSE.txt` |
82
+ | GitHub API | repository exists; `"license": null`; `/license` returns **HTTP 404** |
83
+ | README | **no match** — no licence section |
84
+
85
+ **About the one licence file in the tree.** `SoftPool/LICENSE.txt` is the MIT
86
+ licence of a **vendored third-party dependency**, `alexandrosstergiou/SoftPool`,
87
+ `Copyright (c) 2020 Alexandros Stergiou`. It is reproduced here as
88
+ `LICENSE-SoftPool-MIT.txt` because that dependency's code is genuinely part of
89
+ what the checkpoint was trained with (E3Diff's CPEN module imports SoftPool
90
+ unconditionally). It is **not** a licence for E3Diff, and it must not be read
91
+ as one.
92
+
93
+ **Lineage.** E3Diff's README credits SR3
94
+ (`Janspiry/Image-Super-Resolution-via-Iterative-Refinement`, Apache-2.0),
95
+ `GaParmar/img2img-turbo` (MIT) and `alexandrosstergiou/SoftPool` (MIT). Those
96
+ licences cover the borrowed parts only, not E3Diff's own two-stage contribution.
97
+ We do not ship the SR3 or img2img-turbo licence texts here because neither file
98
+ is vendored in the tree we trained from; fetch them from those repositories if
99
+ you need them.
100
+
101
+ **These are our own retrained weights**, never the authors' released checkpoint.
102
+
103
+ **Conclusion: no licence. Weights published by decision, with this disclosure.**
104
+
105
+ ---
106
+
107
+ ## DDPM (SR3-class)
108
+
109
+ | test | result |
110
+ |---|---|
111
+ | upstream | **the same code base as E3Diff**, `https://github.com/DeepSARRS/E3Diff`, run in its stage-1 configuration (`"stage": 1`) |
112
+ | pinned commit | `38601093ab8f8e4b478144621f20890b100a3b74` |
113
+ | all four tests | **identical results to the E3Diff row above** |
114
+
115
+ This row reproduces the **SR3 method class** — an eps-prediction conditional
116
+ DDPM on the source image — using E3Diff's stage-1 code. It is **not** the SR3
117
+ authors' implementation and no code from the SR3 release was used. Its licence
118
+ position is therefore exactly E3Diff's: none.
119
+
120
+ **Conclusion: no licence. Weights published by decision, with this disclosure.**
121
+
122
+ ---
123
+
124
+ ## Conditional Diffusion (CondDiff)
125
+
126
+ *Conditional Diffusion for SAR to Optical Image Translation*, Bai, Pu and Xu,
127
+ IEEE GRSL, doi:10.1109/LGRS.2023.3337143.
128
+
129
+ | test | result |
130
+ |---|---|
131
+ | upstream | `https://github.com/Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation` |
132
+ | pinned commit | **not recoverable — see below** |
133
+ | filesystem, maxdepth 3 | **no match** — no LICENSE, LICENCE, COPYING or NOTICE anywhere in the vendored tree |
134
+ | tracked files | **test does not apply**: the vendored copy is not a git clone (no `.git`, no `.gitmodules`, no VERSION file) |
135
+ | GitHub API | repository exists; `"license": null`; `/license` returns **HTTP 404** |
136
+ | README | **no match** — no licence section |
137
+
138
+ **A second, separate reproducibility gap.** Because the vendored copy carries no
139
+ version control and records no upstream URL inside the tree, **the exact upstream
140
+ commit this checkpoint was trained from cannot be recovered.** The repository
141
+ URL above is recorded in our own collection script, not in the vendored code. A
142
+ "clone upstream, then apply our patch" recipe is therefore not possible for this
143
+ row; the vendored tree itself is the only source of truth.
144
+
145
+ **Lineage.** CondDiff's README states the repository is based on
146
+ `openai/guided-diffusion` with modifications, and the vendored tree is visibly
147
+ that code base — OpenAI provenance comments survive in
148
+ `guided_diffusion/logger.py` and `guided_diffusion/unet.py`.
149
+ `openai/guided-diffusion` is **MIT**. The unmodified guided-diffusion parts carry
150
+ that MIT licence; it does not extend to the authors' modifications. We do not
151
+ ship the guided-diffusion MIT text here because it is not vendored in the tree we
152
+ trained from — fetch it from that repository if you need it.
153
+
154
+ **We modified the code further**, and say so as a matter of discipline rather
155
+ than because any licence compels it:
156
+
157
+ * the distributed-init helper no longer overwrites `CUDA_VISIBLE_DEVICES`;
158
+ * **the sampler sorts its directory listings.** Upstream paired SAR with EO by
159
+ *unsorted* `os.listdir` position, i.e. by filesystem order, so any number
160
+ produced with the unpatched sampler is measured against effectively arbitrary
161
+ ground truth. This is a correctness fix, not a convenience;
162
+ * the noise tensor for a partial last batch is shaped from the batch, not from
163
+ the `--batch_size` flag;
164
+ * the sampler takes `--test_dir` / `--out_dir` and builds its resize transform at
165
+ run time, so the 512 px cell is not silently resized to 256.
166
+
167
+ All four changes are commented in place in the vendored source. Sampling uses
168
+ respaced DDPM with 250 steps, which is the authors' own `sample.sh` protocol;
169
+ the released DDIM path is broken upstream (`ddim_sample_loop` has no `condition`
170
+ parameter, so passing `--use_ddim True` raises `TypeError`).
171
+
172
+ **Conclusion: no licence. Weights published by decision, with this disclosure.**
173
+
174
+ ---
175
+
176
+ ## Summary
177
+
178
+ | method | upstream | licence file | GitHub `license` | verdict |
179
+ |---|---|---|---|---|
180
+ | StegoGAN | `sian-wusidi/StegoGAN` | none (0 of 52 tracked) | `null`, `/license` 404 | **no grant** |
181
+ | E3Diff | `DeepSARRS/E3Diff` | only a dependency's (`SoftPool/LICENSE.txt`, MIT) | `null`, `/license` 404 | **no grant** |
182
+ | DDPM (SR3-class) | same repo, stage 1 | as above | `null`, `/license` 404 | **no grant** |
183
+ | CondDiff | `Coordi777/Conditional-Diffusion-…` | none | `null`, `/license` 404 | **no grant** |
184
+
185
+ Every other upstream that **does** publish a licence file has its text in this
186
+ same directory, one file per method. The one remaining method, ControlNet, is
187
+ trained with `huggingface/diffusers`, whose grant is an Apache-2.0 header in the
188
+ trainer source rather than a repository LICENSE file, so there is no file to
189
+ copy; that row's weight terms are the Open RAIL++-M text at the repository root.
baselines/qxs-saropt/bbdm/README.md ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BBDM — QXS-SAROPT
2
+
3
+ **BBDM: Image-to-Image Translation with Brownian Bridge Diffusion Models**, Li,
4
+ Xue, Liu and Lai, *CVPR 2023*
5
+ ([arXiv:2205.07680](https://arxiv.org/abs/2205.07680)). Upstream code:
6
+ [`xuekt98/BBDM`](https://github.com/xuekt98/BBDM) at commit
7
+ `02c3b13c9f9dfab0853e32123100680a0640c4ed`.
8
+
9
+ This is the latent Brownian-bridge diffusion model we retrained ourselves on
10
+ QXS-SAROPT, and it is the checkpoint behind the BBDM row of the ReFlowSET
11
+ comparison table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `last_model.pth` | 2,118,317,140 | the whole trained system — see below |
18
+
19
+ `last_model.pth` is a pickled dictionary, not a bare state dict. It holds:
20
+
21
+ * `['model']` — the full `LatentBrownianBridgeModel` state dict: the denoising
22
+ UNet **and**, embedded inside it, the frozen vq-f4 VQGAN submodule;
23
+ * `['ema']` — the EMA shadow weights. **Sampling uses these**, not `['model']`;
24
+ the runner calls `apply_ema()` before generating;
25
+ * `['step']` and `['epoch']`.
26
+
27
+ UNet: latent resolution 64×64, `model_channels` 128, `num_res_blocks` 2,
28
+ attention at resolutions 32/16/8, `channel_mult` (1, 4, 8), 8 heads,
29
+ 64 channels per head, scale-shift normalisation, resblock up/downsampling.
30
+
31
+ The denoising UNet has `condition_key: nocond` — there is no separate
32
+ conditioning branch, because in BBDM the SAR image *is* one endpoint of the
33
+ Brownian bridge and the EO image is the other. The translation is the bridge,
34
+ not a conditioner.
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | **generator updates released** | **50,000** |
41
+ | batch size / resolution | 32 @ 256 px |
42
+ | iterations per epoch | 500 |
43
+ | epochs | 100 |
44
+ | optimizer | Adam, lr 1e-4, β₁ 0.9, no weight decay; `ReduceLROnPlateau` (factor 0.5, patience 3,000, min lr 5e-7) |
45
+ | EMA | decay 0.995, started at step 30,000, updated every 8 steps |
46
+ | bridge | `mt_type` linear, objective `grad`, L1 loss, T = 1,000, `max_var` 1.0 |
47
+ | sampler (test) | 200 skip-sampled steps, η = 1.0, `clip_denoised` off |
48
+ | latent space | CompVis **vq-f4** VQGAN, frozen (`embed_dim` 3, `n_embed` 8192) |
49
+ | horizontal flip | disabled |
50
+
51
+ The step cap and the epoch count coincide exactly here.
52
+
53
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
54
+
55
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
56
+ |---|---|---|---|---|
57
+ | 76.6 | 0.270 | 0.568 | 0.352 | 15.34 |
58
+
59
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
60
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
61
+
62
+ These are our own re-evaluation numbers, measured by us on the images this
63
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
64
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
65
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
66
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
67
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
68
+ convention gives a systematically different number and the two must never be
69
+ mixed, or compared against a paper that used the other one.
70
+
71
+ ## Load it and translate one SAR image
72
+
73
+ There is no smaller entry point than the repository's own runner, because the
74
+ model is constructed with a VQGAN whose weights are loaded from a path in the
75
+ config **even though the same weights are already inside `last_model.pth`**.
76
+ You must have the CompVis latent-diffusion **vq-f4** checkpoint
77
+ (`model.ckpt`, 756,175,527 bytes) on disk and pointed at by
78
+ `model.VQGAN.params.ckpt_path`, or construction fails before any of this runs.
79
+
80
+ ```bash
81
+ # from the BBDM repository root, with a config whose data.dataset_config
82
+ # points at your SAR / EO directories and whose VQGAN ckpt_path exists:
83
+ python main.py -c configs/<your>-LBBDM-f4.yaml --gpu_ids 0 \
84
+ -r <output dir> --sample_to_eval --resume_model last_model.pth
85
+ ```
86
+
87
+ `--resume_model` sets `model.model_load_path`; the runner loads it with
88
+ `torch.load(..., weights_only=False)`, restores the EMA shadow and applies it
89
+ before sampling.
90
+
91
+ Outputs land in
92
+ `<output dir>/<dataset_name>/LBBDM-f4/sample_to_eval/`,
93
+ in three sibling directories: **`200/` holds the generated EO** (the name is the
94
+ `sample_step` count), `condition/` holds the SAR inputs and `ground_truth/` holds
95
+ the targets. Score `200/`.
96
+
97
+ ## Read before using this checkpoint
98
+
99
+ * **Sampling must use the EMA weights.** `['model']` and `['ema']` are both in
100
+ the file and they are different networks. The reported numbers are the EMA.
101
+ * **You need the external vq-f4 VQGAN.** It is not redistributed here — it is
102
+ CompVis latent-diffusion material with its own (MIT) licence, and the config
103
+ requires it at construction time regardless.
104
+ * **`torch.load` needs `weights_only=False`.** PyTorch ≥ 2.6 flipped that
105
+ default; these are pickled Lightning-style checkpoints. We made that change in
106
+ four places in the repository (`model/VQGAN/vqgan.py`, `runners/BaseRunner.py`
107
+ twice, `runners/.../BBDMRunner.py`) and also removed a `verbose=True` kwarg
108
+ from `ReduceLROnPlateau` that PyTorch 2.x no longer accepts. All four are
109
+ compatibility fixes; none changes the maths.
110
+ * Passes the identity-collapse audit on both datasets.
111
+
112
+ ## Licence — stated factually
113
+
114
+ Upstream `xuekt98/BBDM` ships a LICENSE file, reproduced verbatim here as
115
+ `licenses/LICENSE-bbdm.txt`: the **MIT licence**, `Copyright (c) 2023 xuekt98`.
116
+
117
+ **Redistribution of these weights is permitted.** No non-commercial clause, no
118
+ share-alike clause.
119
+
120
+ One dependency note that is attribution, not restriction: this checkpoint cannot
121
+ be run without the CompVis latent-diffusion **vq-f4** VQGAN, which is MIT and is
122
+ **not** redistributed here.
123
+
124
+ Attribution: *trained with `xuekt98/BBDM` (MIT, Copyright (c) 2023 xuekt98).
125
+ Requires the CompVis latent-diffusion vq-f4 VQGAN (MIT,
126
+ https://github.com/CompVis/latent-diffusion), which is not redistributed here.
127
+ Full text in `licenses/LICENSE-bbdm.txt`.*
128
+
129
+ ---
130
+
131
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
132
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
133
+ itself. Every comparison method in this repository was retrained by us on the
134
+ same splits at the same resolution and scored through one evaluation pipeline,
135
+ so the rows are directly comparable to each other — and, for the same reason,
136
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/cbbdm/README.md ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # cBBDM — QXS-SAROPT
2
+
3
+ **Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image
4
+ Translation**, Kim and Chung, *IEEE Geoscience and Remote Sensing Letters*, 2025
5
+ ([arXiv:2408.07947](https://arxiv.org/abs/2408.07947); the venue and year are
6
+ taken from the authors' own citation block — the repository records no DOI).
7
+ Upstream code:
8
+ [`egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical`](https://github.com/egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical)
9
+ at commit `8ce15934f4d4e3f01efe70d11e2d9b9e0859210c`, a fork of
10
+ [`xuekt98/BBDM`](https://github.com/xuekt98/BBDM).
11
+
12
+ This is the conditional Brownian-bridge latent diffusion model we retrained
13
+ ourselves on QXS-SAROPT, and it is the checkpoint behind the cBBDM row of the
14
+ ReFlowSET comparison table.
15
+
16
+ ## What is in this folder
17
+
18
+ | file | bytes | what it is |
19
+ |---|---|---|
20
+ | `last_model.pth` | 2,118,345,622 | the whole trained system — see below |
21
+
22
+ `last_model.pth` is a pickled dictionary, not a bare state dict. It holds:
23
+
24
+ * `['model']` — the full `LatentBrownianBridgeModel` state dict: the denoising
25
+ UNet **and**, embedded inside it, the frozen vq-f4 VQGAN submodule;
26
+ * `['ema']` — the EMA shadow weights. **Sampling uses these**, not `['model']`;
27
+ the runner calls `apply_ema()` before generating;
28
+ * `['step']` and `['epoch']`.
29
+
30
+ UNet: latent resolution 64×64, `model_channels` 128, `num_res_blocks` 2,
31
+ attention at resolutions 32/16/8, `channel_mult` (1, 4, 8), 8 heads,
32
+ 64 channels per head, scale-shift normalisation, resblock up/downsampling.
33
+
34
+ The denoising UNet differs from plain BBDM in exactly two ways, and they are
35
+ the method: **`in_channels` is 6, not 3**, and **`condition_key` is
36
+ `SpatialRescaler`** — a small trained module that rescales the SAR latent and
37
+ concatenates it to the bridge state at every step, so the SAR conditioning is
38
+ explicit rather than implicit in the bridge endpoints. The SpatialRescaler is
39
+ trained (it is in the model's parameter list) and is inside this file.
40
+
41
+ ## Training budget we used
42
+
43
+ | | |
44
+ |---|---|
45
+ | **generator updates released** | **50,000** |
46
+ | batch size / resolution | 32 @ 256 px |
47
+ | iterations per epoch | 500 |
48
+ | epochs | 100 |
49
+ | optimizer | Adam, lr 1e-4, β₁ 0.9, no weight decay; `ReduceLROnPlateau` (factor 0.5, patience 3,000, min lr 5e-7) |
50
+ | EMA | decay 0.995, started at step 30,000, updated every 8 steps |
51
+ | bridge | `mt_type` linear, objective `grad`, L1 loss, T = 1,000, `max_var` 1.0 |
52
+ | sampler (test) | 200 skip-sampled steps, η = 1.0, `clip_denoised` off |
53
+ | latent space | CompVis **vq-f4** VQGAN, frozen (`embed_dim` 3, `n_embed` 8192) |
54
+ | horizontal flip | disabled |
55
+
56
+ The step cap and the epoch count coincide exactly here.
57
+
58
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
59
+
60
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
61
+ |---|---|---|---|---|
62
+ | 50.6 | 0.246 | 0.539 | 0.372 | 16.02 |
63
+
64
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
65
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
66
+
67
+ These are our own re-evaluation numbers, measured by us on the images this
68
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
69
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
70
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
71
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
72
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
73
+ convention gives a systematically different number and the two must never be
74
+ mixed, or compared against a paper that used the other one.
75
+
76
+ ## Load it and translate one SAR image
77
+
78
+ There is no smaller entry point than the repository's own runner, because the
79
+ model is constructed with a VQGAN whose weights are loaded from a path in the
80
+ config **even though the same weights are already inside `last_model.pth`**.
81
+ You must have the CompVis latent-diffusion **vq-f4** checkpoint
82
+ (`model.ckpt`, 756,175,527 bytes) on disk and pointed at by
83
+ `model.VQGAN.params.ckpt_path`, or construction fails before any of this runs.
84
+
85
+ ```bash
86
+ # from the cBBDM repository root, with a config whose data.dataset_config
87
+ # points at your SAR / EO directories and whose VQGAN ckpt_path exists:
88
+ python main.py -c configs/<your>-cBBDM-f4.yaml --gpu_ids 0 \
89
+ -r <output dir> --sample_to_eval --resume_model last_model.pth
90
+ ```
91
+
92
+ `--resume_model` sets `model.model_load_path`; the runner loads it with
93
+ `torch.load(..., weights_only=False)`, restores the EMA shadow and applies it
94
+ before sampling.
95
+
96
+ Outputs land in
97
+ `<output dir>/<dataset_name>/cBBDM-f4/sample_to_eval/`,
98
+ in three sibling directories: **`200/` holds the generated EO** (the name is the
99
+ `sample_step` count), `condition/` holds the SAR inputs and `ground_truth/` holds
100
+ the targets. Score `200/`.
101
+
102
+ ## Read before using this checkpoint
103
+
104
+ * **Sampling must use the EMA weights.** `['model']` and `['ema']` are both in
105
+ the file and they are different networks. The reported numbers are the EMA.
106
+ * **You need the external vq-f4 VQGAN.** It is not redistributed here — it is
107
+ CompVis latent-diffusion material with its own (MIT) licence, and the config
108
+ requires it at construction time regardless.
109
+ * **`torch.load` needs `weights_only=False`.** PyTorch ≥ 2.6 flipped that
110
+ default; these are pickled Lightning-style checkpoints. We made that change in
111
+ four places in the repository (`model/VQGAN/vqgan.py`, `runners/BaseRunner.py`
112
+ twice, `runners/.../BBDMRunner.py`) and also removed a `verbose=True` kwarg
113
+ from `ReduceLROnPlateau` that PyTorch 2.x no longer accepts. All four are
114
+ compatibility fixes; none changes the maths.
115
+ * **A performance patch we applied, and when.** This fork regressed the
116
+ DataLoader worker count from upstream BBDM's 8 to 0, which dominated wall-clock
117
+ time (one 32-pair batch: 6.5 s serial versus 1.1 s across 8 threads). We
118
+ restored `num_workers=8`. **Both cells published here were trained *before*
119
+ that change, i.e. with 0 workers** — same data, same order, same step count,
120
+ only slower. The numbers on this page are unaffected either way: worker count
121
+ cannot change which samples the model sees.
122
+ * Passes the identity-collapse audit on both datasets.
123
+
124
+ ## Licence — stated factually
125
+
126
+ Upstream `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical` ships a LICENSE file,
127
+ reproduced verbatim here as `licenses/LICENSE-cbbdm.txt`: the **MIT licence**,
128
+ `Copyright (c) 2025 egshkim`. Its README adds: *"The original source code is
129
+ licensed under the MIT License, and this repository continues to comply with its
130
+ terms"* — the original being `xuekt98/BBDM`, also MIT
131
+ (`licenses/LICENSE-bbdm.txt`).
132
+
133
+ **Redistribution of these weights is permitted.** No non-commercial clause, no
134
+ share-alike clause.
135
+
136
+ One dependency note that is attribution, not restriction: this checkpoint cannot
137
+ be run without the CompVis latent-diffusion **vq-f4** VQGAN, which is MIT and is
138
+ **not** redistributed here.
139
+
140
+ Attribution: *trained with `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical`
141
+ (MIT, Copyright (c) 2025 egshkim), a fork of `xuekt98/BBDM` (MIT, Copyright (c)
142
+ 2023 xuekt98). Requires the CompVis vq-f4 VQGAN (MIT,
143
+ https://github.com/CompVis/latent-diffusion), not redistributed here. Full texts
144
+ in `licenses/LICENSE-cbbdm.txt` and `licenses/LICENSE-bbdm.txt`.*
145
+
146
+ ---
147
+
148
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
149
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
150
+ itself. Every comparison method in this repository was retrained by us on the
151
+ same splits at the same resolution and scored through one evaluation pipeline,
152
+ so the rows are directly comparable to each other — and, for the same reason,
153
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/cdiffset/README.md ADDED
@@ -0,0 +1,164 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # C-DiffSET — QXS-SAROPT
2
+
3
+ **C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with
4
+ Confidence-Guided Reliable Object Generation**, Do, Lee, Lee and Kim,
5
+ *IEEE Transactions on Circuits and Systems for Video Technology*, 2026
6
+ ([doi:10.1109/TCSVT.2026.3701447](https://doi.org/10.1109/TCSVT.2026.3701447),
7
+ [arXiv:2411.10788](https://arxiv.org/abs/2411.10788)). Upstream code:
8
+ [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET).
9
+
10
+ This is the **stage-2 (confidence-guided) UNet** we retrained ourselves on
11
+ QXS-SAROPT, and it is the checkpoint behind the C-DiffSET row of the ReFlowSET
12
+ comparison table.
13
+
14
+ ## What is in this folder
15
+
16
+ A single `UNet2DConditionModel` (865,925,125 parameters, fp32, 686 tensors) with
17
+ two channel modifications relative to stock SD 2.1-base:
18
+
19
+ * **`conv_in` 4 -> 8 channels.** The UNet is conditioned by channel
20
+ concatenation in latent space, **SAR first**:
21
+ `torch.cat([sar_latent, noisy_eo_latent], dim=1)`. The extra half was
22
+ initialised by duplicating the pretrained kernel and halving it; in this
23
+ trained checkpoint the two halves have clearly diverged
24
+ (mean |w| 0.01424 on channels 0-3 vs 0.02810 on channels 4-7).
25
+ * **`conv_out` 4 -> 5 channels.** Channels 0-3 are the usual epsilon
26
+ prediction; channel 4 is a **raw spatial variance** used by the
27
+ confidence-guided loss. It was zero-initialised and is no longer zero
28
+ (mean |w| 0.02922), i.e. the head really trained. To turn it into a variance:
29
+ `var = clamp(softplus(raw + c) + 1e-6, 1e-6, 10)` where the offset
30
+ `c = log(exp(1 - 1e-6) - 1)` is the inverse softplus of the initial variance,
31
+ and confidence is `1 / var`. **The scheduler must only ever see channels 0-3.**
32
+
33
+ There is no VAE, no text encoder and no scheduler here — take those from the
34
+ base model (below).
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | optimizer updates released | **40,000** (fixed-step snapshot `checkpoint-40000`) |
41
+ | updates the run was configured for | 50,000 |
42
+ | initialised from | our own stage-1 checkpoint (the `sd21ft` folder's run, at its final 50,000-update state) |
43
+ | batch size / resolution | 64 @ 256 px |
44
+ | epochs the 40,000 updates work out to | ~160 (16,001 training pairs, 250 updates/epoch) |
45
+ | optimizer | AdamW, lr 3e-5, weight decay 0.01, cosine schedule, 100-step warmup |
46
+ | precision | fp32 (no mixed precision) |
47
+ | seed | 2024 |
48
+ | prediction type | epsilon, with the confidence-guided (Gaussian-NLL) loss |
49
+ | prompt | the fixed string `"electro-optical image"` |
50
+ | augmentation | horizontal flip, vertical flip, 90-degree rotation |
51
+ | test-time sampler | **DDIM, 50 steps**, fp32, one image at a time |
52
+
53
+ The budget is quoted in **optimizer updates**, not epochs, because the two
54
+ datasets have very different training-set sizes. Two things a reader
55
+ reproducing this needs to know: the released snapshot is taken at update 40,000
56
+ of a cosine schedule that was laid out for 50,000, so the learning rate had
57
+ **not** finished decaying at that point; and we publish the fixed-step snapshot
58
+ rather than the repository's validation-selected `best/`, because every other
59
+ method in our comparison publishes its last checkpoint and selecting on
60
+ validation PSNR would have been an asymmetry in C-DiffSET's favour.
61
+
62
+ Data: 256 px, no crop and no resize (the chips are already 256 px). SAR chips come from the `sar_256_oc_0.2` directory of QXS-SAROPT; the EO target is the same stem under `opt_256_oc_0.2`.
63
+
64
+ ## Base model
65
+
66
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
67
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
68
+ available on the Hub, our training and every snippet below load the community
69
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
70
+ That is what we actually trained against — verified in the training
71
+ configuration files, the launch scripts and the training logs, none of which
72
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
73
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
74
+ expected to work but has not been verified by us.
75
+
76
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
77
+
78
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
79
+ |---|---|---|---|---|---|
80
+ | 19.9 | 0.0055 | 0.233 | 0.526 | 16.92 | 0.380 |
81
+
82
+ These are our own re-evaluation numbers, not numbers copied from any paper.
83
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
84
+ convention); the alternative [0, 1] convention gives a systematically different
85
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
86
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
87
+
88
+ ## Load it and translate one SAR PNG
89
+
90
+ ```python
91
+ import numpy as np
92
+ import torch
93
+ from PIL import Image
94
+ from diffusers import UNet2DConditionModel, AutoencoderKL, DDIMScheduler
95
+ from transformers import CLIPTextModel, CLIPTokenizer
96
+
97
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
98
+ BASE = "Manojb/stable-diffusion-2-1-base" # frozen VAE / text encoder / scheduler
99
+ DEV, DT = "cuda", torch.float32
100
+
101
+ unet = UNet2DConditionModel.from_pretrained(
102
+ REPO, subfolder="baselines/qxs-saropt/cdiffset", torch_dtype=DT).to(DEV).eval()
103
+ vae = AutoencoderKL.from_pretrained(BASE, subfolder="vae", torch_dtype=DT).to(DEV).eval()
104
+ tok = CLIPTokenizer.from_pretrained(BASE, subfolder="tokenizer")
105
+ txt = CLIPTextModel.from_pretrained(BASE, subfolder="text_encoder", torch_dtype=DT).to(DEV).eval()
106
+ sch = DDIMScheduler.from_pretrained(BASE, subfolder="scheduler")
107
+ sch.set_timesteps(50, device=DEV)
108
+
109
+ # The fixed training prompt, tokenized exactly as in training (no padding).
110
+ ids = tok("electro-optical image", padding="do_not_pad",
111
+ max_length=tok.model_max_length, truncation=True,
112
+ return_tensors="pt").input_ids.to(DEV)
113
+ with torch.no_grad():
114
+ embed = txt(ids)[0]
115
+
116
+ # SAR input: 256x256 PNG -> RGB -> [-1, 1]. Do not resize; crop if needed.
117
+ sar = Image.open("sar.png").convert("RGB")
118
+ x = torch.from_numpy(np.array(sar)).permute(2, 0, 1)[None]
119
+ x = (x.to(DEV, DT) / 255.0) * 2 - 1
120
+
121
+ with torch.no_grad():
122
+ sar_lat = vae.encode(x).latent_dist.mean * vae.config.scaling_factor # .mean, not .sample()
123
+ eo_lat = torch.randn_like(sar_lat)
124
+ for t in sch.timesteps:
125
+ out = unet(torch.cat([sar_lat, eo_lat], 1), # SAR first, then the noisy EO latent
126
+ t, encoder_hidden_states=embed).sample
127
+ # channels 0-3 are epsilon; channel 4 is the raw variance and never
128
+ # goes to the scheduler.
129
+ eo_lat = sch.step(out[:, :4], t, eo_lat).prev_sample
130
+ eo = vae.decode(eo_lat / vae.config.scaling_factor).sample
131
+ eo = (eo * 0.5 + 0.5).clamp(0, 1)[0].cpu()
132
+
133
+ Image.fromarray((eo.permute(1, 2, 0).numpy() * 255).round().astype(np.uint8)).save("eo.png")
134
+ ```
135
+
136
+ The three things that silently produce a plausible-but-wrong image if you get
137
+ them backwards: the concatenation order is **SAR first**, only `out[:, :4]`
138
+ reaches the scheduler, and the VAE must be read as `latent_dist.mean`, not
139
+ `.sample()`.
140
+
141
+ ## Licence status — stated factually
142
+
143
+ * **Training code:** MIT, `Copyright (c) 2026 KAIST VICLab`
144
+ ([`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET) ships a
145
+ LICENSE file).
146
+ * **These weights:** they are a fine-tune of Stable Diffusion 2.1-base and are
147
+ therefore a *Derivative of the Model* under the **CreativeML Open RAIL++-M**
148
+ licence. The base repository we loaded declares `openrail++`.
149
+ The **Attachment A use restrictions travel with these weights** — they bind
150
+ you, and you must pass them on to anyone you redistribute to. The full text,
151
+ including Attachment A, ships in this repository as `LICENSE-WEIGHTS.md`.
152
+
153
+ Attribution: *fine-tuned from Stable Diffusion 2.1-base (obtained as
154
+ `Manojb/stable-diffusion-2-1-base`, a re-upload of
155
+ `stabilityai/stable-diffusion-2-1-base`) with the training code of
156
+ `KAIST-VICLab/C-DiffSET`. Do, Lee, Lee and Kim, "C-DiffSET", IEEE TCSVT 2026,
157
+ doi:10.1109/TCSVT.2026.3701447.*
158
+
159
+ ---
160
+
161
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
162
+ checkpoint retrained by us on QXS-SAROPT; it is not ReFlowSET itself. Every
163
+ comparison method in the release was retrained by us on the same splits at the
164
+ same resolution, so the rows are directly comparable to each other.
baselines/qxs-saropt/cdiffset/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "UNet2DConditionModel",
3
+ "_diffusers_version": "0.10.0.dev0",
4
+ "act_fn": "silu",
5
+ "attention_head_dim": [
6
+ 5,
7
+ 10,
8
+ 20,
9
+ 20
10
+ ],
11
+ "block_out_channels": [
12
+ 320,
13
+ 640,
14
+ 1280,
15
+ 1280
16
+ ],
17
+ "center_input_sample": false,
18
+ "cross_attention_dim": 1024,
19
+ "down_block_types": [
20
+ "CrossAttnDownBlock2D",
21
+ "CrossAttnDownBlock2D",
22
+ "CrossAttnDownBlock2D",
23
+ "DownBlock2D"
24
+ ],
25
+ "downsample_padding": 1,
26
+ "dual_cross_attention": false,
27
+ "flip_sin_to_cos": true,
28
+ "freq_shift": 0,
29
+ "in_channels": 8,
30
+ "layers_per_block": 2,
31
+ "mid_block_scale_factor": 1,
32
+ "norm_eps": 1e-05,
33
+ "norm_num_groups": 32,
34
+ "num_class_embeds": null,
35
+ "only_cross_attention": false,
36
+ "out_channels": 5,
37
+ "sample_size": 64,
38
+ "up_block_types": [
39
+ "UpBlock2D",
40
+ "CrossAttnUpBlock2D",
41
+ "CrossAttnUpBlock2D",
42
+ "CrossAttnUpBlock2D"
43
+ ],
44
+ "use_linear_projection": true
45
+ }
baselines/qxs-saropt/conddiff/README.md ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Conditional Diffusion — QXS-SAROPT
2
+
3
+ **Conditional Diffusion for SAR to Optical Image Translation**, Bai, Pu and Xu,
4
+ *IEEE Geoscience and Remote Sensing Letters*, 2023
5
+ ([doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143); the
6
+ year is the one in the authors' own citation block — a table that labels this
7
+ row with a later issue year is referring to the same paper).
8
+
9
+ Upstream code:
10
+ [`Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation`](https://github.com/Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation),
11
+ a modified copy of [`openai/guided-diffusion`](https://github.com/openai/guided-diffusion).
12
+ **The exact upstream commit is not recoverable** — see the licence section.
13
+
14
+ This is the pixel-space conditional diffusion model we retrained ourselves on
15
+ QXS-SAROPT, and it is the checkpoint behind the Conditional Diffusion row of the
16
+ ReFlowSET comparison table.
17
+
18
+ ## What is in this folder
19
+
20
+ | file | bytes | what it is |
21
+ |---|---|---|
22
+ | `ema_final.pt` | 657,495,287 | EMA (decay 0.9999) of the UNet at update 50,000 — a **bare state dict**, no wrapper key |
23
+
24
+ `ema_final.pt` is the EMA of a **164.3 M-parameter guided-diffusion UNet**:
25
+ `num_channels` 128, `num_res_blocks` 3, `learn_sigma` **False**, attention at
26
+ resolutions 16 and 8. It is a plain `state_dict` — load it directly, with no
27
+ `['model']` or `['ema']` indirection.
28
+
29
+ The QXS-SAROPT and SAR2Opt files differ in size (657,495,287 vs 662,459,343
30
+ bytes) purely because of the 256 px versus 512 px positional and attention
31
+ buffers; the parameter count is otherwise the same.
32
+
33
+ **Conditioning.** The SAR image is concatenated to the noisy state **noise-free**
34
+ at every reverse step and at training time — that is the paper's claim, and the
35
+ code does exactly that.
36
+
37
+ ## Training budget we used
38
+
39
+ | | |
40
+ |---|---|
41
+ | **generator updates released** | **50,000** |
42
+ | batch size / resolution | 24 @ 256 px |
43
+ | optimizer | Adam, **constant** lr 1e-4 with `--lr_anneal_steps 50000` (linear decay to zero, which is also the only stop mechanism in the released code) |
44
+ | EMA | decay 0.9999 |
45
+ | diffusion | T = 2,000, linear β schedule, eps-prediction, `learn_sigma` False |
46
+ | sampler (test) | **respaced DDPM, 250 steps**, `clip_denoised` on |
47
+ | augmentation | none — the released code has none |
48
+ | input | the chips are natively 256 px, used as they are |
49
+
50
+ **The paper and the released code disagree about the learning-rate schedule.**
51
+ The paper describes warmup plus cosine; the released code implements neither. We
52
+ ran the code.
53
+
54
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
55
+
56
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
57
+ |---|---|---|---|---|
58
+ | 88.6 | 0.355 | 0.730 | 0.213 | 11.55 |
59
+
60
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
61
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
62
+
63
+ These are our own re-evaluation numbers, measured by us on the images this
64
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
65
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
66
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
67
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
68
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
69
+ convention gives a systematically different number and the two must never be
70
+ mixed, or compared against a paper that used the other one.
71
+
72
+ ## Load it and translate one SAR image
73
+
74
+ ```python
75
+ import torch
76
+ from guided_diffusion.script_util import (create_model_and_diffusion,
77
+ model_and_diffusion_defaults)
78
+
79
+ d = model_and_diffusion_defaults()
80
+ d.update(image_size=256, num_channels=128, num_res_blocks=3, learn_sigma=False,
81
+ diffusion_steps=2000, noise_schedule='linear', timestep_respacing='250')
82
+ model, diffusion = create_model_and_diffusion(**d)
83
+ model.load_state_dict(torch.load('ema_final.pt', map_location='cpu'))
84
+ model.cuda().eval()
85
+
86
+ # sar: (1, 3, 256, 256) float tensor in [-1, 1]
87
+ sample = diffusion.p_sample_loop(model, (1, 3, 256, 256),
88
+ clip_denoised=True, model_kwargs={},
89
+ noise=None, condition=sar)
90
+ ```
91
+
92
+ The repository imports `blobfile` and `mpi4py` unconditionally; a single-process
93
+ run on a local filesystem needs either those packages or small local stand-ins on
94
+ `PYTHONPATH`.
95
+
96
+ **Never pass `use_ddim=True`.** See below.
97
+
98
+ ## Read before using this checkpoint
99
+
100
+ * **DDIM is broken upstream, and that is not a choice we made.**
101
+ `p_sample_loop(..., condition=None, ...)` accepts and threads the SAR
102
+ condition; `ddim_sample_loop(...)` has **no `condition` parameter at all**, so
103
+ passing `--use_ddim True` raises `TypeError`. Sampling is respaced DDPM with
104
+ 250 steps, which is also what the authors' own `sample.sh` uses.
105
+ * **A correctness bug in the released sampler, which we fixed.** Upstream paired
106
+ each SAR image with an EO image by **unsorted `os.listdir` position** — i.e. by
107
+ filesystem order. Any number produced with the unpatched sampler is measured
108
+ against effectively arbitrary ground truth. We sort both listings. If you
109
+ reproduce this row from a clean upstream checkout, apply that fix or your
110
+ metrics are meaningless.
111
+ * **Three further changes we made**, all commented in place: the
112
+ distributed-init helper no longer overwrites `CUDA_VISIBLE_DEVICES` (upstream
113
+ pins rank % 8, which on a shared machine hijacks another user's device); the
114
+ noise tensor for a partial last batch is shaped from the batch rather than from
115
+ the `--batch_size` flag; and the sampler takes explicit input/output directories
116
+ and builds its resize transform at run time, because the module-level transform
117
+ hard-codes 256 px and would silently downsize the 512 px cell.
118
+ * **The training loader requires integer filenames.** It sorts with
119
+ `int(stem)`, so any non-numeric stem raises `ValueError`. Feed it an
120
+ integer-named adapter directory and keep a manifest to map back to the real
121
+ stems.
122
+ * **Do not let the released 512 px path resize.** The fork's `center_crop_arr`
123
+ *resizes* 600 → 512. We wrote deterministic centre-512 crops instead, to keep a
124
+ crop-not-resize protocol across the whole benchmark.
125
+ * Passes the identity-collapse audit on both datasets.
126
+
127
+ ## Licence — stated factually ⚠ no upstream licence exists
128
+
129
+ **The upstream code base publishes no licence.**
130
+ [`Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation`](https://github.com/Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation)
131
+ has no LICENSE, LICENCE, COPYING or NOTICE file anywhere in the tree we trained
132
+ from, no licence section in its README, and the GitHub API reports no declared
133
+ licence, with the `/license` endpoint returning 404. Checked 2026-08-28.
134
+
135
+ Under default copyright that means **all rights are reserved by the authors and
136
+ no express permission to redistribute derived work has been granted** to us or
137
+ to you. We publish this checkpoint anyway and state the position plainly. Assess
138
+ redistribution for yourself; consider asking the authors.
139
+
140
+ **A second gap, which is about reproducibility rather than licensing.** Our
141
+ vendored copy of this repository carries no version control and records no
142
+ upstream URL inside its tree, so **the exact commit these weights were trained
143
+ from cannot be recovered**. A "clone upstream, then apply our patch" recipe is
144
+ therefore not possible for this row.
145
+
146
+ **Lineage.** The README states the repository is based on
147
+ `openai/guided-diffusion` with modifications, and the tree is visibly that code
148
+ base — OpenAI provenance comments survive in `guided_diffusion/logger.py` and
149
+ `guided_diffusion/unet.py`. `openai/guided-diffusion` is **MIT**; the unmodified
150
+ guided-diffusion parts carry that licence, which does **not** extend to the
151
+ authors' modifications. We do not ship the guided-diffusion licence text here
152
+ because it is not vendored in the tree we trained from — take it from that
153
+ repository if you need it, and make sure the copyright line you carry is
154
+ OpenAI's rather than another project's MIT file.
155
+
156
+ **We modified the code further**, and say so as a matter of discipline rather
157
+ than because any licence compels it: the four changes listed in the section
158
+ above, of which the sorted-listing fix is a correctness fix.
159
+
160
+ Please cite: Bai, Pu and Xu, *Conditional Diffusion for SAR to Optical Image
161
+ Translation*, IEEE Geoscience and Remote Sensing Letters,
162
+ [doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143).
163
+ The full record of what we checked, per method, is in
164
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
165
+
166
+ ---
167
+
168
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
169
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
170
+ itself. Every comparison method in this repository was retrained by us on the
171
+ same splits at the same resolution and scored through one evaluation pipeline,
172
+ so the rows are directly comparable to each other — and, for the same reason,
173
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/controlnet/README.md ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ControlNet — QXS-SAROPT
2
+
3
+ **Adding Conditional Control to Text-to-Image Diffusion Models**, Zhang, Rao and
4
+ Agrawala, *ICCV* 2023. The SAR image is the control signal and the base
5
+ Stable Diffusion 2.1-base UNet stays frozen.
6
+
7
+ > **How our row was produced, precisely.** We trained with the
8
+ > `examples/controlnet/train_controlnet.py` script from
9
+ > [`huggingface/diffusers`](https://github.com/huggingface/diffusers)
10
+ > (Apache-2.0, Copyright 2025 The HuggingFace Inc. team), initialising the
11
+ > adapter with `ControlNetModel.from_unet` on SD 2.1-base. **No code from
12
+ > `lllyasviel/ControlNet` was used.** The method is theirs; the implementation
13
+ > is the diffusers one.
14
+
15
+ ## What is in this folder
16
+
17
+ A complete diffusers `ControlNetModel` folder — `config.json` plus
18
+ `diffusion_pytorch_model.safetensors` (364,228,240 parameters, fp32, 340
19
+ tensors). It is the encoder half of the UNet plus the zero-convolutions and the
20
+ conditioning embedder; it has no `up_blocks` and no `conv_out`, so **it cannot
21
+ generate anything on its own** and must be paired with the base pipeline.
22
+
23
+ The conditioning input is 3-channel RGB (`conditioning_channels: 3`) with
24
+ embedder widths `[16, 32, 96, 256]`; the SAR PNG is opened as RGB, so the three
25
+ channels are identical.
26
+
27
+ ## Training budget we used
28
+
29
+ | | |
30
+ |---|---|
31
+ | optimizer updates | **50,000** |
32
+ | batch size / resolution | 32 @ 256 px |
33
+ | epochs the 50,000 updates work out to | ~100 (16,001 training pairs, 500 updates/epoch) |
34
+ | optimizer | AdamW, lr 1e-5, **constant** schedule (the script's default) |
35
+ | precision | bf16 mixed precision — the saved tensors are the fp32 master weights |
36
+ | seed | 42 |
37
+ | trainable parameters | the adapter only; the SD 2.1-base UNet, VAE and text encoder stayed frozen |
38
+ | prompt | the fixed string `"electro-optical image"` for every pair |
39
+ | augmentation | **none** — the diffusers trainer applies only `Resize(res)` + `CenterCrop(res)`, both no-ops at our input size, and no flips or rotations |
40
+ | test-time sampler | **UniPCMultistepScheduler, 50 steps, guidance scale 7.5**, bf16 |
41
+
42
+ Unlike the two UNet folders in this repository, this is **not** an SD 2.1
43
+ fine-tune: the base weights were never updated, which is why only the 364 M
44
+ adapter ships.
45
+
46
+ Reproduction note: our inference script creates **one** `torch.Generator`
47
+ seeded 42 and reuses it across the whole test set in batches of 32,
48
+ so a single image regenerated on its own will not be bit-identical to our dump
49
+ even at the same seed. The conditioning image is
50
+ the SAR PNG opened as RGB and bilinearly resized to 256x256 (a no-op at our
51
+ evaluation resolution, since the SAR inputs are already 256 px).
52
+
53
+ Data: 256 px chips, used as-is (no crop, no resize). SAR chips come from the `sar_256_oc_0.2` directory of QXS-SAROPT; the EO target is the same stem under `opt_256_oc_0.2`.
54
+
55
+ ## Base model
56
+
57
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
58
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
59
+ available on the Hub, our training and every snippet below load the community
60
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
61
+ That is what we actually trained against — verified in the training
62
+ configuration files, the launch scripts and the training logs, none of which
63
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
64
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
65
+ expected to work but has not been verified by us.
66
+
67
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
68
+
69
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
70
+ |---|---|---|---|---|---|
71
+ | 50.4 | 0.0211 | 0.307 | 0.604 | 13.42 | 0.297 |
72
+
73
+ These are our own re-evaluation numbers, not numbers copied from any paper.
74
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
75
+ convention); the alternative [0, 1] convention gives a systematically different
76
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
77
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
78
+
79
+ ## Load it and translate one SAR PNG
80
+
81
+ ```python
82
+ import torch
83
+ from PIL import Image
84
+ from diffusers import (ControlNetModel, StableDiffusionControlNetPipeline,
85
+ UniPCMultistepScheduler)
86
+
87
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
88
+ BASE = "Manojb/stable-diffusion-2-1-base"
89
+
90
+ controlnet = ControlNetModel.from_pretrained(
91
+ REPO, subfolder="baselines/qxs-saropt/controlnet", torch_dtype=torch.bfloat16)
92
+ pipe = StableDiffusionControlNetPipeline.from_pretrained(
93
+ BASE, controlnet=controlnet, torch_dtype=torch.bfloat16,
94
+ safety_checker=None).to("cuda")
95
+ pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
96
+
97
+ sar = Image.open("sar.png").convert("RGB").resize((256, 256), Image.BILINEAR)
98
+ eo = pipe("electro-optical image", image=sar, height=256, width=256,
99
+ num_inference_steps=50, guidance_scale=7.5,
100
+ generator=torch.Generator("cuda").manual_seed(42)).images[0]
101
+ eo.save("eo.png")
102
+ ```
103
+
104
+ ## Licence status — stated factually
105
+
106
+ * **Training code:** Apache-2.0. The trainer is
107
+ `examples/controlnet/train_controlnet.py` from
108
+ [`huggingface/diffusers`](https://github.com/huggingface/diffusers),
109
+ `Copyright 2025 The HuggingFace Inc. team`; the Apache-2.0 grant is in the
110
+ file header. Our copy of that script is four loose files rather than a full
111
+ clone, so it carries no separate repository LICENSE file — the header is the
112
+ grant.
113
+ * **Method credit:** ControlNet (Zhang et al., ICCV 2023),
114
+ [`lllyasviel/ControlNet`](https://github.com/lllyasviel/ControlNet),
115
+ Apache-2.0. **No code from that repository was used in this row.**
116
+ * **These weights:** initialised from Stable Diffusion 2.1-base via
117
+ `ControlNetModel.from_unet`, so they are an SD 2.1 derivative under the
118
+ **CreativeML Open RAIL++-M** licence (`openrail++`). The **Attachment A use
119
+ restrictions travel with these weights**. Full text, including Attachment A,
120
+ ships in this repository as `LICENSE-WEIGHTS.md`.
121
+
122
+ ---
123
+
124
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
125
+ checkpoint retrained by us on QXS-SAROPT; it is not ReFlowSET itself. Every
126
+ comparison method in the release was retrained by us on the same splits at the
127
+ same resolution, so the rows are directly comparable to each other.
baselines/qxs-saropt/controlnet/config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "ControlNetModel",
3
+ "_diffusers_version": "0.37.1",
4
+ "act_fn": "silu",
5
+ "addition_embed_type": null,
6
+ "addition_embed_type_num_heads": 64,
7
+ "addition_time_embed_dim": null,
8
+ "attention_head_dim": [
9
+ 5,
10
+ 10,
11
+ 20,
12
+ 20
13
+ ],
14
+ "block_out_channels": [
15
+ 320,
16
+ 640,
17
+ 1280,
18
+ 1280
19
+ ],
20
+ "class_embed_type": null,
21
+ "conditioning_channels": 3,
22
+ "conditioning_embedding_out_channels": [
23
+ 16,
24
+ 32,
25
+ 96,
26
+ 256
27
+ ],
28
+ "controlnet_conditioning_channel_order": "rgb",
29
+ "cross_attention_dim": 1024,
30
+ "down_block_types": [
31
+ "CrossAttnDownBlock2D",
32
+ "CrossAttnDownBlock2D",
33
+ "CrossAttnDownBlock2D",
34
+ "DownBlock2D"
35
+ ],
36
+ "downsample_padding": 1,
37
+ "encoder_hid_dim": null,
38
+ "encoder_hid_dim_type": null,
39
+ "flip_sin_to_cos": true,
40
+ "freq_shift": 0,
41
+ "global_pool_conditions": false,
42
+ "in_channels": 4,
43
+ "layers_per_block": 2,
44
+ "mid_block_scale_factor": 1,
45
+ "mid_block_type": "UNetMidBlock2DCrossAttn",
46
+ "norm_eps": 1e-05,
47
+ "norm_num_groups": 32,
48
+ "num_attention_heads": null,
49
+ "num_class_embeds": null,
50
+ "only_cross_attention": false,
51
+ "projection_class_embeddings_input_dim": null,
52
+ "resnet_time_scale_shift": "default",
53
+ "transformer_layers_per_block": 1,
54
+ "upcast_attention": false,
55
+ "use_linear_projection": true
56
+ }
baselines/qxs-saropt/cyclegan/README.md ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CycleGAN — QXS-SAROPT
2
+
3
+ **Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial
4
+ Networks**, Zhu, Park, Isola and Efros, *ICCV 2017*
5
+ ([arXiv:1703.10593](https://arxiv.org/abs/1703.10593)). Upstream code:
6
+ [`junyanz/pytorch-CycleGAN-and-pix2pix`](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
7
+ at commit `2a7afba2895d52556dd5dfe07e8555ef657ced6f`.
8
+
9
+ This is the unpaired cycle-consistent GAN we retrained ourselves on QXS-SAROPT,
10
+ and it is the checkpoint behind the CycleGAN row of the ReFlowSET comparison
11
+ table. **It fails our input-copy audit — see the section below before you use
12
+ or quote it.**
13
+
14
+ ## What is in this folder
15
+
16
+ | file | bytes | what it is |
17
+ |---|---|---|
18
+ | `net_G_A.pth` | 45,533,133 | **SAR → EO** — this is the translation direction the table reports |
19
+ | `net_G_B.pth` | 45,533,133 | EO → SAR, the reverse generator that closes the cycle; not used to produce any reported number |
20
+
21
+ Both are **`resnet_9blocks` generators** with instance normalisation,
22
+ `ngf = 64`, 3 in / 3 out, dropout disabled. The direction assignment is not a
23
+ convention we chose: upstream's unaligned loader binds `A_paths` to the
24
+ `trainA` / `testA` directory, which holds the SAR side, and the model computes
25
+ `fake_B = netG_A(real_A)`. So **`net_G_A` is the SAR → EO generator**.
26
+
27
+ `net_G_B` is shipped because it is half of the trained system and someone
28
+ reproducing the cycle needs it; it is not needed to reproduce our numbers. The
29
+ two discriminators are not released.
30
+
31
+ ## Training budget we used
32
+
33
+ | | |
34
+ |---|---|
35
+ | **generator updates released** | **100,050** |
36
+ | batch size | 8 |
37
+ | epochs (constant LR + linear decay) | 25 + 25 |
38
+ | iterations per epoch | 2,001 (16,001 training images, the loader does not drop the last batch) |
39
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
40
+ | adversarial loss | LSGAN; discriminator `basic` (70×70 PatchGAN) |
41
+ | cycle losses | λ_A 10, λ_B 10 |
42
+ | identity loss | λ_identity 0.5 |
43
+ | augmentation | horizontal flip |
44
+ | input | 256 px, no resize |
45
+
46
+ Budgets are quoted in **optimizer updates**, never epochs. This is the single
47
+ most important comparability rule for the unpaired methods in this benchmark:
48
+ "100 + 100 epochs" is 72,600 updates on 1,450 images and 400,000+ on
49
+ 16,001, a difference that reads as budget-matched and is not.
50
+
51
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
52
+
53
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
54
+ |---|---|---|---|---|
55
+ | 104.4 | 0.376 | 0.653 | 0.262 | 12.92 |
56
+
57
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
58
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
59
+
60
+ These are our own re-evaluation numbers, measured by us on the images this
61
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
62
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
63
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
64
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
65
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
66
+ convention gives a systematically different number and the two must never be
67
+ mixed, or compared against a paper that used the other one.
68
+
69
+ ## Load it and translate one SAR image
70
+
71
+ Clone `junyanz/pytorch-CycleGAN-and-pix2pix` at the commit above and run this
72
+ from its root. The model code is stock — we applied no patch to it.
73
+
74
+ ```python
75
+ import numpy as np, torch
76
+ from PIL import Image
77
+ from models.networks import define_G # from the junyanz repo
78
+
79
+ G_A = define_G(3, 3, 64, 'resnet_9blocks', 'instance', use_dropout=False,
80
+ init_type='normal', init_gain=0.02, gpu_ids=[])
81
+ sd = torch.load('net_G_A.pth', map_location='cpu')
82
+ if hasattr(sd, '_metadata'):
83
+ del sd._metadata
84
+ G_A.load_state_dict(sd)
85
+ G_A.eval()
86
+
87
+ sar = Image.open('sar.png').convert('RGB')
88
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
89
+
90
+ with torch.no_grad():
91
+ y = G_A(x) # this is fake_B
92
+
93
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
94
+ Image.fromarray(eo).save('eo.png')
95
+ ```
96
+
97
+ Upstream also supports a SAR-only run without this snippet:
98
+ `test.py --model test --dataset_mode single --no_dropout --model_suffix _A`,
99
+ which loads `latest_net_G_A.pth` directly.
100
+
101
+ ## ⚠ This cell fails the input-copy audit
102
+
103
+ We audit every cell for **identity collapse**: a model that has learned to pass
104
+ its input through rather than translate it. The test compares the mean absolute
105
+ distance from the generated image to the SAR **input** against the distance to
106
+ the EO **target**; a ratio below 1.0 means the output is closer to the input it
107
+ was given than to the answer it was asked for.
108
+
109
+ | quantity | value |
110
+ |---|---|
111
+ | mean \|generated − ground truth\| | 47.548 |
112
+ | mean \|generated − SAR input\| | 40.274 |
113
+ | **ratio** (SAR distance ÷ GT distance) | **0.847** — below 1.0 |
114
+ | verdict | **COLLAPSED** |
115
+ | cross-pair separation | 0.955 (cohort median 1.160) — no oracle behaviour |
116
+
117
+ **Read the metrics on this page with that in mind.** They are reported unchanged
118
+ because they are what the released implementation does at its own published
119
+ protocol, and a benchmark that quietly substitutes a better-behaved variant is
120
+ not reporting the method. But they should not be read as translation quality.
121
+ The separation figure is the companion check for the *opposite* failure: it is
122
+ the mean distance from a generated image to **other** images' ground truth
123
+ divided by the distance to its **own**, so a value well above the cohort median
124
+ would indicate that ground-truth information had leaked into the generation.
125
+ This cell sits at or below the median, so there is no leak — the problem here is
126
+ input-copying, and only input-copying.
127
+
128
+ On this dataset the collapse is at least partly a budget effect: a diagnostic control run of the same code at **400,000** updates (4× this one) does **not** collapse (ratio 1.127). That control is not published here and is not the table's CycleGAN row, which is the method at its own protocol's budget.
129
+
130
+ ## Read before using this checkpoint
131
+
132
+ * **`net_G_A` is SAR → EO.** Loading `net_G_B` and expecting an EO image gives
133
+ you the reverse mapping and a confusing result.
134
+ * Build the network with `gpu_ids=[]` — the checkpoints have bare keys and a
135
+ non-empty `gpu_ids` introduces a `module.` prefix the load will reject.
136
+ * Normalisation is `[-1, 1]`.
137
+ * CycleGAN is **unpaired**: it never saw a SAR/EO correspondence during
138
+ training, only the two marginal distributions.
139
+
140
+ ## Licence — stated factually
141
+
142
+ Same upstream, same LICENSE file as pix2pix; reproduced verbatim here as
143
+ `licenses/LICENSE-cyclegan.txt` (byte-identical to `licenses/LICENSE-pix2pix.txt`
144
+ — one file covers both methods upstream). It is three stacked notices:
145
+
146
+ * CycleGAN — `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park` (BSD-2-Clause in
147
+ form);
148
+ * pix2pix — `Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu`;
149
+ * DCGAN — `Copyright (c) 2015, Facebook, Inc.` (BSD-3-Clause, with a
150
+ no-endorsement clause).
151
+
152
+ **Redistribution of these weights is permitted** with the notice reproduced. No
153
+ non-commercial and no share-alike clause. Do not use Facebook's name to promote
154
+ this release.
155
+
156
+ Attribution: *trained with `junyanz/pytorch-CycleGAN-and-pix2pix`. Copyright (c)
157
+ 2017, Jun-Yan Zhu and Taesung Park (CycleGAN); Copyright (c) 2016, Phillip Isola
158
+ and Jun-Yan Zhu (pix2pix); Copyright (c) 2015, Facebook, Inc. (DCGAN). All rights
159
+ reserved. Redistributed under the BSD terms reproduced in
160
+ `licenses/LICENSE-cyclegan.txt`.*
161
+
162
+ ---
163
+
164
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
165
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
166
+ itself. Every comparison method in this repository was retrained by us on the
167
+ same splits at the same resolution and scored through one evaluation pipeline,
168
+ so the rows are directly comparable to each other — and, for the same reason,
169
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/ddpm/README.md ADDED
@@ -0,0 +1,174 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # DDPM (SR3-class) — QXS-SAROPT
2
+
3
+ The **SR3 method class** — an eps-prediction conditional DDPM that conditions on
4
+ the source image — as introduced in *Image Super-Resolution via Iterative
5
+ Refinement*, Saharia, Ho, Chan, Salimans, Fleet and Norouzi,
6
+ *IEEE TPAMI 2022* ([arXiv:2104.07636](https://arxiv.org/abs/2104.07636)).
7
+
8
+ **Read this before citing the row: the code is not the SR3 authors'.** No code
9
+ from any SR3 release was used. This cell is
10
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) at commit
11
+ `38601093ab8f8e4b478144621f20890b100a3b74`, run in its **stage-1**
12
+ configuration (`"stage": 1`), which is an SR3-class conditional DDPM. We
13
+ retrained it ourselves on QXS-SAROPT and it is the checkpoint behind the DDPM
14
+ row of the ReFlowSET comparison table.
15
+
16
+ ## What is in this folder
17
+
18
+ | file | bytes | what it is |
19
+ |---|---|---|
20
+ | `gen.pth` | 768,426,944 | the `sr3` UNet at iteration 250,000 — the whole model |
21
+
22
+ `gen.pth` is E3Diff's **`sr3` UNet**: `inner_channel` 64,
23
+ `channel_multiplier` [1, 2, 4, 8, 16], `res_blocks` 1, `norm_groups` 16, **no
24
+ attention** (`attn_res` is empty), 3 input channels, 3 output channels and a
25
+ 3-channel condition.
26
+
27
+ The condition is **not the raw SAR image alone**. It is a three-channel stack
28
+ `[PPB, Canny, SAR]`:
29
+
30
+ * channel 0 — the SAR image after **FAST_PPB** speckle filtering
31
+ (Deledalle 2009, P = 3, W = 10, h = 0.5), a GPU port of E3Diff's own
32
+ `FAST_PPB.m`;
33
+ * channel 1 — `cv2.Canny(ppb_uint8, 50, 150, L2gradient=True)` on that filtered
34
+ image;
35
+ * channel 2 — the raw SAR image.
36
+
37
+ **This row cannot be run from a SAR PNG alone**: the PPB and Canny images must
38
+ be computed first.
39
+
40
+ ## Training budget we used
41
+
42
+ | | |
43
+ |---|---|
44
+ | **generator updates released** | **250,000** |
45
+ | batch size / resolution | 16 @ 256 px |
46
+ | optimizer | Adam, lr 5e-5, constant (`MultiStepLR` with no milestones) |
47
+ | EMA | decay 0.9999, started at step 5,000, updated every step |
48
+ | diffusion (train) | T = 1,000, linear β from 1e-6 to 1e-2 |
49
+ | sampler (test) | **DDIM, 50 steps** |
50
+ | loss | plain eps-prediction MSE — the stage-2 auxiliary losses are all zero here (`lpips_w` 0, `fft_w` 0, `lambda_gan` 0) |
51
+ | augmentation | flips, 90° rotations, and a random brightness jitter applied to the SAR/PPB pair only (upstream's own `transform_augment`) |
52
+
53
+ Both datasets get the same 250,000 updates; the batch differs so that the
54
+ pixels-per-update budget matches (16 × 256² = 4 × 512²).
55
+
56
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
57
+
58
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
59
+ |---|---|---|---|---|
60
+ | 43.8 | 0.311 | 0.620 | 0.359 | 14.04 |
61
+
62
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
63
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
64
+
65
+ These are our own re-evaluation numbers, measured by us on the images this
66
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
67
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
68
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
69
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
70
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
71
+ convention gives a systematically different number and the two must never be
72
+ mixed, or compared against a paper that used the other one.
73
+
74
+ ## Load it and translate one SAR image
75
+
76
+ This model cannot be driven from a bare SAR file: it needs the PPB and Canny
77
+ condition channels, and the E3Diff loader looks them up by filename. Build the
78
+ condition first, then run the repository's own entry point.
79
+
80
+ ```python
81
+ # 1. Build the condition channels for each SAR image, once.
82
+ import cv2, numpy as np
83
+ # ppb = FAST_PPB(sar, P=3, W=10, h=0.5) # Deledalle 2009; E3Diff ships FAST_PPB.m
84
+ # canny = cv2.Canny(ppb.astype(np.uint8), 50, 150, L2gradient=True)
85
+ # Write ppb and canny under <root>/val/SAR-PPB/<name> and <root>/val/SAR-canny/<name>,
86
+ # alongside <root>/val/SAR/<name> and <root>/val/EO/<name>. Names must match exactly.
87
+
88
+ # 2. Run the repository, phase 'val', with a config whose
89
+ # path.resume_state is the checkpoint PREFIX -- no '_gen.pth' suffix:
90
+ # "path": {"resume_state": "<dir>/I250000_E<epoch>"} # place gen.pth as I250000_E<epoch>_gen.pth
91
+ # "stage": 1,
92
+ # "model": {"beta_schedule": {"val": {"n_timestep": 50, "ddim": 1, ...}}},
93
+ # "datasets": {"val": {"data_len": -1, "r_resolution": 256}}
94
+ # python main.py -c <val.json> -p val -enable_wandb "" --seed 1
95
+ ```
96
+
97
+ Two mechanical details that will otherwise cost you a full inference pass:
98
+
99
+ * the RGB loader. Upstream's `SAR2EODataset` hard-codes a **grayscale** EO
100
+ target and a two-channel `[PPB, Canny]` condition. QXS-SAROPT's optical side is
101
+ RGB, and with three channels upstream's `ddim_sample()` slices the condition to
102
+ `condition_x[:, :3]` and concatenates it with 3-channel predictions, which
103
+ raises on the first validation image. We ran through a small wrapper that
104
+ patches `SAR2EODataset.__getitem__` **in memory** — returning
105
+ `HR = EO[0:3]`, `LR = SAR[0:3]`, `SR = cat(PPB[0:1], Canny[0:1], SAR[0:1])` —
106
+ and then executes the repository's `main.py` verbatim, so no upstream file is
107
+ touched and every flag behaves the same. Reproduce that patch or write your own
108
+ three-channel loader;
109
+ * `-enable_wandb ""` is mandatory. The flag defaults to the string `'false'`,
110
+ which is truthy.
111
+
112
+ ## Read before using this checkpoint
113
+
114
+ * **The checkpoint loads with `strict=False`.** `model/model.py` calls
115
+ `load_state_dict(..., strict=False)`, so a configuration that does not match
116
+ the file **loads nothing and raises no error** — you get a fully random model
117
+ and a plausible-looking run. Verify the `Loading pretrained model for G [...]`
118
+ line in the log and eyeball the first output.
119
+ * **`-p val` writes next to the checkpoint and then renames.** Images land in
120
+ `<resume_state>/sample/`, and at the end the directory is renamed to
121
+ `<resume_state>_S<ssim>_P<psnr>_l2<l2>_Lp<lpips>`. That rename **raises if the
122
+ target already exists** — after the entire inference has been paid for. Guard
123
+ for an existing `<prefix>_S*/sample` before starting.
124
+ * **SoftPool.** The E3Diff code imports the SoftPool CUDA extension
125
+ unconditionally. We used a pure-PyTorch drop-in with identical maths
126
+ (`softpool(x) = avgpool(x·eˣ) / avgpool(eˣ)`) and identical autograd, which
127
+ needs no build step. Either build the extension or write the equivalent.
128
+ * **A known tone offset, inherited deliberately.** On another dataset in the
129
+ wider campaign this exact configuration produced generations that were, on
130
+ average, tens of gray levels brighter than the ground truth, at a substantial
131
+ cost in PSNR. We did **not** correct it per dataset, because comparability
132
+ across the row was judged worth more than per-cell tuning. Read this row's
133
+ PSNR with that in mind.
134
+ * Passes the identity-collapse audit on both datasets.
135
+
136
+ ## Licence — stated factually ⚠ no upstream licence exists
137
+
138
+ **The upstream code base publishes no licence.**
139
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff), whose stage-1
140
+ configuration this row is, has no LICENSE, LICENCE, COPYING or NOTICE file at
141
+ any depth in its tree; only 1 of its 58 tracked files is licence-shaped and that
142
+ one is `SoftPool/LICENSE.txt`, the MIT licence of a **vendored third-party
143
+ dependency** (`Copyright (c) 2020 Alexandros Stergiou`, reproduced here as
144
+ `licenses/LICENSE-SoftPool-MIT.txt`) — not a grant for E3Diff. Its README has no
145
+ licence section, and the GitHub API reports no declared licence, with the
146
+ `/license` endpoint returning 404. Checked 2026-08-28.
147
+
148
+ Under default copyright that means **all rights are reserved by the authors and
149
+ no express permission to redistribute derived work has been granted** to us or
150
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
151
+ end to end, and we state the position plainly rather than implying a permission
152
+ that does not exist. If you intend to redistribute this checkpoint or build on
153
+ it, assess that for yourself, and consider asking the authors directly.
154
+
155
+ The code base credits SR3
156
+ (`Janspiry/Image-Super-Resolution-via-Iterative-Refinement`, Apache-2.0),
157
+ `GaParmar/img2img-turbo` (MIT) and `alexandrosstergiou/SoftPool` (MIT). Those
158
+ licences cover the borrowed parts, not the authors' own contribution.
159
+
160
+ Please cite the SR3 paper for the method class, and E3Diff for the code that
161
+ produced these weights: Qin, Zou, Li and Zhang, *Efficient End-to-End Diffusion
162
+ Model for One-Step SAR-to-Optical Translation*, IEEE Geoscience and Remote
163
+ Sensing Letters, [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).
164
+ The full record of what we checked, per method, is in
165
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
166
+
167
+ ---
168
+
169
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
170
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
171
+ itself. Every comparison method in this repository was retrained by us on the
172
+ same splits at the same resolution and scored through one evaluation pipeline,
173
+ so the rows are directly comparable to each other — and, for the same reason,
174
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/e3diff/README.md ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # E3Diff — QXS-SAROPT
2
+
3
+ **Efficient End-to-End Diffusion Model for One-Step SAR-to-Optical
4
+ Translation**, Qin, Zou, Li and Zhang, *IEEE Geoscience and Remote Sensing
5
+ Letters*, 2024
6
+ ([doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566); the
7
+ year is the one in the authors' own citation block — a table that labels this row
8
+ with a later issue year is referring to the same paper). Upstream code:
9
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) at commit
10
+ `38601093ab8f8e4b478144621f20890b100a3b74`.
11
+
12
+ This is the authors' two-stage method, **retrained by us end to end** on
13
+ QXS-SAROPT, and it is the checkpoint behind the E3Diff row of the ReFlowSET
14
+ comparison table.
15
+
16
+ **These are our weights, not the authors' released checkpoint.** The authors'
17
+ own weights exist for a different dataset and are not what this row measures.
18
+
19
+ ## What is in this folder
20
+
21
+ | file | bytes | what it is |
22
+ |---|---|---|
23
+ | `gen.pth` | 768,426,944 | the `sr3` UNet after stage 2, at absolute iteration 310,000 |
24
+
25
+ Architecturally this is **the same network as the DDPM (SR3-class) row** in this
26
+ repository — E3Diff's two stages share one UNet. What differs is the training:
27
+
28
+ * **stage 1** (250,000 updates) is the eps-prediction conditional DDPM. Its
29
+ checkpoint is the `ddpm/` folder in this repository.
30
+ * **stage 2** (60,000 further updates, absolute iteration 310,000) fine-tunes
31
+ that same network into a **one-step generator**: the sampler is run *with
32
+ gradients* from pure noise for `ddim_steps = 1`, and the loss is taken directly
33
+ on pixels.
34
+
35
+ UNet: `inner_channel` 64, `channel_multiplier` [1, 2, 4, 8, 16], `res_blocks` 1,
36
+ `norm_groups` 16, no attention, 3 in / 3 out, 3-channel condition.
37
+
38
+ The condition is the same three-channel `[PPB, Canny, SAR]` stack described on
39
+ the DDPM card: FAST_PPB speckle filtering (Deledalle 2009, P = 3, W = 10,
40
+ h = 0.5), `cv2.Canny(ppb, 50, 150, L2gradient=True)`, and the raw SAR image.
41
+ **This row cannot be run from a SAR PNG alone.**
42
+
43
+ ## Training budget we used
44
+
45
+ | | |
46
+ |---|---|
47
+ | **generator updates released** | **250,000 (stage 1, inherited) + 60,000 (stage 2) = 310,000 absolute** |
48
+ | batch size / resolution | 16 @ 256 px |
49
+ | optimizer | Adam, lr 5e-5 |
50
+ | stage-2 sampler during training | DDIM, **1 step**, run with gradients from pure noise |
51
+ | stage-2 losses | L1 on pixels, + LPIPS (weight 5), + focal-frequency (weight 10), + a vision-aided CLIP GAN (λ_gan 0.5) |
52
+ | stage-1 losses | plain eps-prediction MSE (all auxiliary weights zero) |
53
+ | sampler (test) | **DDIM, 1 step** |
54
+ | EMA | decay 0.9999 (stage 1) |
55
+ | augmentation | flips, 90° rotations, and a random brightness jitter on the SAR/PPB pair (upstream's `transform_augment`) |
56
+
57
+ The stage-2 budget is 24 % of the stage-1 budget, which follows the authors' own
58
+ ratio (they resume at 640,000 and run to 800,000). Note that `n_iter` in this
59
+ code base is an **absolute** step count continuing stage 1, not a stage-2
60
+ budget — a config that reads 310,000 buys 60,000 new iterations.
61
+
62
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
63
+
64
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
65
+ |---|---|---|---|---|
66
+ | 47.8 | 0.278 | 0.530 | 0.302 | 16.44 |
67
+
68
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
69
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
70
+
71
+ These are our own re-evaluation numbers, measured by us on the images this
72
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
73
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
74
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
75
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
76
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
77
+ convention gives a systematically different number and the two must never be
78
+ mixed, or compared against a paper that used the other one.
79
+
80
+ ## Load it and translate one SAR image
81
+
82
+ Same entry point and same mechanics as the DDPM row, with `"stage": 2` and
83
+ `ddim_steps: 1`.
84
+
85
+ ```python
86
+ # 1. Build the condition channels for each SAR image, once:
87
+ # ppb = FAST_PPB(sar, P=3, W=10, h=0.5)
88
+ # canny = cv2.Canny(ppb.astype('uint8'), 50, 150, L2gradient=True)
89
+ # written to <root>/val/SAR-PPB/<name> and <root>/val/SAR-canny/<name>,
90
+ # alongside <root>/val/SAR/<name> and <root>/val/EO/<name>.
91
+
92
+ # 2. Run phase 'val' with a config whose path.resume_state is the checkpoint
93
+ # PREFIX -- no '_gen.pth' suffix -- and place gen.pth as <prefix>_gen.pth:
94
+ # "stage": 2, "ddim_steps": 1,
95
+ # "model": {"beta_schedule": {"val": {"n_timestep": 1, "ddim": 1, ...}}},
96
+ # "datasets": {"val": {"data_len": -1, "r_resolution": 256}}
97
+ # python main.py -c <val.json> -p val -enable_wandb "" --seed 1
98
+ ```
99
+
100
+ Sampling is genuinely one network evaluation per image, which is the point of
101
+ the method.
102
+
103
+ As on the DDPM card, the RGB loader is the one thing you must sort out: upstream
104
+ hard-codes a grayscale EO target and a two-channel condition, and with three
105
+ channels its `ddim_sample()` slices the condition and concatenates mismatched
106
+ shapes. We patched `SAR2EODataset.__getitem__` in memory to return
107
+ `HR = EO[0:3]`, `LR = SAR[0:3]`, `SR = cat(PPB[0:1], Canny[0:1], SAR[0:1])` and
108
+ then ran the repository's `main.py` verbatim.
109
+
110
+ ## Read before using this checkpoint
111
+
112
+ * **This is not an oracle.** Worth stating, because a one-step generator invites
113
+ the question: the sampler's `condition_x` is the `[PPB, Canny, SAR]` stack, and
114
+ the ground truth never enters it. The cell also passes our leak audit on both
115
+ datasets.
116
+ * **Stage 2 needs the *real* vision-aided CLIP discriminator.** Stage 1 has
117
+ `lambda_gan = 0` and can import a stub; stage 2 cannot. If a constant-output
118
+ stub shadows the real `vision_aided_loss` package, stage 2 trains against a
119
+ constant GAN loss and **silently is not the authors' method**. Assert on
120
+ `vision_aided_loss.__file__` before starting a run.
121
+ * **SoftPool.** The code imports the SoftPool CUDA extension unconditionally. We
122
+ used a pure-PyTorch drop-in with identical maths
123
+ (`softpool(x) = avgpool(x·eˣ) / avgpool(eˣ)`) and identical autograd, which
124
+ needs no build step.
125
+ * **The checkpoint loads with `strict=False`.** A mismatched configuration
126
+ **loads nothing and raises no error**. Verify the `Loading pretrained model for
127
+ G [...]` log line and look at the first output.
128
+ * **`-p val` writes next to the checkpoint and then renames** the sample
129
+ directory to `<prefix>_S<ssim>_P<psnr>_l2<l2>_Lp<lpips>`. That rename raises if
130
+ the target already exists — after the whole inference has been paid for.
131
+ * **`-enable_wandb ""` is mandatory**; the flag defaults to the truthy string
132
+ `'false'`.
133
+ * One change we made to the repository: `core/logger.py` no longer
134
+ unconditionally overwrites `CUDA_VISIBLE_DEVICES` from the config's `gpu_ids`,
135
+ which on a shared machine remapped every job onto another user's device. It is
136
+ a scheduling fix and does not affect the model.
137
+
138
+ ## Licence — stated factually ⚠ no upstream licence exists
139
+
140
+ **The upstream code base publishes no licence.**
141
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) has no LICENSE,
142
+ LICENCE, COPYING or NOTICE file at the repository root or at any depth; **1 of
143
+ its 58 tracked files** is licence-shaped, and it is `SoftPool/LICENSE.txt`, the
144
+ MIT licence of a **vendored third-party dependency** (`Copyright (c) 2020
145
+ Alexandros Stergiou`, reproduced here as `licenses/LICENSE-SoftPool-MIT.txt`) —
146
+ not a grant covering E3Diff. Its README has no licence section, and the GitHub
147
+ API reports no declared licence, with the `/license` endpoint returning 404.
148
+ Checked 2026-08-28.
149
+
150
+ Under default copyright that means **all rights are reserved by the authors and
151
+ no express permission to redistribute derived work has been granted** to us or
152
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
153
+ end to end, and we state the position plainly rather than implying a permission
154
+ that does not exist. If you intend to redistribute this checkpoint or build on
155
+ it, assess that for yourself, and consider asking the authors directly.
156
+
157
+ **Lineage.** E3Diff's README credits SR3
158
+ (`Janspiry/Image-Super-Resolution-via-Iterative-Refinement`, Apache-2.0),
159
+ `GaParmar/img2img-turbo` (MIT) and `alexandrosstergiou/SoftPool` (MIT). Those
160
+ licences cover the borrowed parts only, not the authors' own two-stage
161
+ contribution. We ship only the SoftPool text, because it is the only one of the
162
+ three that is vendored in the tree we trained from.
163
+
164
+ Please cite: Qin, Zou, Li and Zhang, *Efficient End-to-End Diffusion Model for
165
+ One-Step SAR-to-Optical Translation*, IEEE Geoscience and Remote Sensing
166
+ Letters, [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).
167
+ The full record of what we checked, per method, is in
168
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
169
+
170
+ ---
171
+
172
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
173
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
174
+ itself. Every comparison method in this repository was retrained by us on the
175
+ same splits at the same resolution and scored through one evaluation pipeline,
176
+ so the rows are directly comparable to each other — and, for the same reason,
177
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/hidiff/README.md ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # HI-Diff — QXS-SAROPT
2
+
3
+ **Hierarchical Integration Diffusion Model for Realistic Image Deblurring**,
4
+ Chen, Zhang, Liu, Kong and Yuan, *NeurIPS 2023*
5
+ ([arXiv:2305.12966](https://arxiv.org/abs/2305.12966)). Upstream code:
6
+ [`zhengchen1999/HI-Diff`](https://github.com/zhengchen1999/HI-Diff) at commit
7
+ `b3bfd167997e27f8edd57681cf70e5031a0e35f2`.
8
+
9
+ This is the two-stage latent-prior restoration transformer we retrained
10
+ ourselves on QXS-SAROPT (SAR as the degraded input, EO as the target), and it is
11
+ the checkpoint behind the HI-Diff row of the ReFlowSET comparison table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `S1_net_g_latest.pth` | 101,401,457 | stage-1 restoration transformer — for reproducing stage-2 *training*, not for inference |
18
+ | `S1_net_le_latest.pth` | 2,357,509 | stage-1 latent encoder, `in_chans` **6** (it sees LQ ‖ GT) — **not usable at test time** |
19
+ | `S2_net_d_latest.pth` | 10,530,491 | **the 8-step latent denoiser** — needed at inference |
20
+ | `S2_net_g_latest.pth` | 101,401,457 | **the final restoration transformer** — needed at inference |
21
+ | `S2_net_le_dm_latest.pth` | 2,247,061 | **the stage-2 latent encoder**, `in_chans` 3 (LQ only) — needed at inference |
22
+
23
+ **Only the three `S2_*` files are needed to run this model.** The repository's
24
+ own test configuration loads exactly `pretrain_network_g`,
25
+ `pretrain_network_le_dm` and `pretrain_network_d`, and nothing else. The two
26
+ `S1_*` files are shipped so that stage-2 *training* can be reproduced: stage 1
27
+ learns a latent prior from the LQ **and** the ground truth together, which is why
28
+ its encoder takes 6 input channels and cannot be used at deployment.
29
+
30
+ Architecture: the transformer is Restormer-shaped — `dim` 48, blocks
31
+ [3, 5, 6, 6], 4 refinement blocks, heads [1, 2, 4, 8], FFN expansion 2.66,
32
+ `embed_dim` 64, `group` 4. The latent denoiser runs **8 timesteps** on the
33
+ compressed prior (`linear_start` 0.1, `linear_end` 0.99) — the diffusion here
34
+ acts on a small latent prior, not on the image.
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | **generator updates released** | **25,000 (stage 1) + 25,000 (stage 2) = 50,000** |
41
+ | batch size / patch | 8 @ 256 px |
42
+ | optimizer | AdamW, lr 2e-4, weight decay 1e-4, β (0.9, 0.999), gradient clipping on |
43
+ | schedule | `CosineAnnealingRestartCyclicLR`, periods [8,000 / 17,000], η_min [2e-4, 1e-6] |
44
+ | loss | L1 |
45
+ | progressive training | **disabled** |
46
+ | test resolution | tested at native 256 px |
47
+
48
+ The upstream recipe is 300,000 + 300,000 iterations with a progressive
49
+ patch/batch schedule from 128 to 384 px. We disabled the progressive schedule
50
+ (a single fixed 256 px group) and matched the budget to the rest of this
51
+ benchmark's diffusion band at 50,000 total updates. That is a substantial
52
+ reduction from the authors' recipe and is stated here so nobody reads this row
53
+ as HI-Diff at full budget.
54
+
55
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
56
+
57
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
58
+ |---|---|---|---|---|
59
+ | 324.3 | 0.539 | 0.692 | 0.457 | 17.10 |
60
+
61
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
62
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
63
+
64
+ These are our own re-evaluation numbers, measured by us on the images this
65
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
66
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
67
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
68
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
69
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
70
+ convention gives a systematically different number and the two must never be
71
+ mixed, or compared against a paper that used the other one.
72
+
73
+ ## Load it and translate one SAR image
74
+
75
+ The three networks must be assembled by hand; the repository's own `test.py`
76
+ with a `HI_Diff_S2` option file is the path of least resistance.
77
+
78
+ ```yaml
79
+ # options/test/<your>.yml -- the three paths below are the three S2_ files
80
+ model_type: HI_Diff_S2
81
+ network_g: {type: Transformer, inp_channels: 3, out_channels: 3, dim: 48,
82
+ num_blocks: [3,5,6,6], num_refinement_blocks: 4, heads: [1,2,4,8],
83
+ ffn_expansion_factor: 2.66, bias: False, LayerNorm_type: WithBias,
84
+ dual_pixel_task: False, embed_dim: 64, group: 4}
85
+ network_le_dm: {type: latent_encoder_gelu, in_chans: 3, embed_dim: 64, block_num: 6,
86
+ group: 4, stage: 2, patch_expansion: 0.5, channel_expansion: 4}
87
+ network_d: {type: denoising, in_channel: 256, out_channel: 256, inner_channel: 512,
88
+ block_num: 4, group: 4, patch_expansion: 0.5, channel_expansion: 2}
89
+ diffusion_schedule: {apply_ldm: False, schedule: linear, timesteps: 8,
90
+ linear_start: 0.1, linear_end: 0.99}
91
+ path:
92
+ pretrain_network_g: S2_net_g_latest.pth
93
+ pretrain_network_le_dm: S2_net_le_dm_latest.pth
94
+ pretrain_network_d: S2_net_d_latest.pth
95
+ param_key_g: params
96
+ ```
97
+
98
+ ```bash
99
+ python test.py -opt options/test/<your>.yml
100
+ ```
101
+
102
+ The SAR input is read with `cv2.IMREAD_COLOR`, so a single-channel SAR image is
103
+ replicated to three channels automatically — that is what training saw. At
104
+ inference `le_dm` produces the prior from the degraded input alone, `net_d`
105
+ denoises that prior over 8 steps, and `net_g` is conditioned on it.
106
+
107
+ ## Read before using this checkpoint
108
+
109
+ * **The `S1_*` files are not an inference path.** `S1_net_le_latest.pth` has
110
+ `in_chans: 6` because stage 1 encodes LQ concatenated with the ground truth.
111
+ Wiring it into a test run would make the model an oracle. Use the `S2_*` trio.
112
+ * **These are the `latest` checkpoints — iteration 25,000 — not a best-validation
113
+ selection.** That is deliberate and benchmark-wide: every other method here
114
+ publishes its last checkpoint, so selecting on validation for this one would
115
+ be an asymmetry in its favour. Do not repoint at a best-val file and then
116
+ compare against this table.
117
+ * **Retraining is destructive.** The upstream launcher path is not idempotent:
118
+ the training configs set `resume_state: ~`, no auto-resume flag is passed, and
119
+ basicsr's `mkdir_and_rename` archives a finished experiment directory and
120
+ starts from scratch. Re-running a training script over a finished experiment
121
+ destroys it.
122
+ * **`torch.load` needs `weights_only=False`** under PyTorch ≥ 2.6; we made that
123
+ change in two places in the repository (`hi_diff/utils/base_model.py` and
124
+ `train.py`). Both are compatibility fixes.
125
+ * **How to read this row's numbers.** HI-Diff takes the best PSNR and SSIM of
126
+ any method in this benchmark on both datasets, while sitting at or near the
127
+ worst FID, DISTS and LPIPS. That combination is the signature of
128
+ regression-to-the-mean: a blurred, low-variance prediction minimises squared
129
+ error and is perceptually poor. Reporting its PSNR without its FID would badly
130
+ misrepresent it — and so would the reverse.
131
+ * Passes the identity-collapse audit on both datasets.
132
+
133
+ ## Licence — stated factually
134
+
135
+ Upstream `zhengchen1999/HI-Diff` ships a LICENSE file, reproduced verbatim here
136
+ as `licenses/LICENSE-hidiff.txt`: the **Apache License 2.0**, appendix copyright
137
+ line `Copyright 2023 HI-Diff Authors`.
138
+
139
+ **Redistribution of these weights is permitted.** Apache-2.0 §4 obligations
140
+ apply: ship the licence, keep the notices, and **state that you modified the
141
+ files**. We did modify them — the two `torch.load(..., weights_only=False)`
142
+ changes noted above, plus our own option files — and this sentence is that
143
+ statement. Upstream ships no NOTICE file, so there is none to carry. Apache-2.0
144
+ §6 withholds trademark rights.
145
+
146
+ HI-Diff itself builds on BasicSR, Restormer and DiffIR, which its README
147
+ credits; those projects' own licences cover their code.
148
+
149
+ Attribution: *trained with `zhengchen1999/HI-Diff`, Copyright 2023 HI-Diff
150
+ Authors, licensed under the Apache License 2.0; our training and test
151
+ configurations modify the upstream configuration, and two source files were
152
+ modified for PyTorch ≥ 2.6 compatibility. HI-Diff builds on BasicSR, Restormer
153
+ and DiffIR. Full text in `licenses/LICENSE-hidiff.txt`.*
154
+
155
+ ---
156
+
157
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
158
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
159
+ itself. Every comparison method in this repository was retrained by us on the
160
+ same splits at the same resolution and scored through one evaluation pipeline,
161
+ so the rows are directly comparable to each other — and, for the same reason,
162
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/p2phd/README.md ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # pix2pixHD — QXS-SAROPT
2
+
3
+ **High-Resolution Image Synthesis and Semantic Manipulation with Conditional
4
+ GANs**, Wang, Liu, Zhu, Tao, Kautz and Catanzaro, *CVPR 2018*
5
+ ([arXiv:1711.11585](https://arxiv.org/abs/1711.11585)). Upstream code:
6
+ [`NVIDIA/pix2pixHD`](https://github.com/NVIDIA/pix2pixHD) at commit
7
+ `14b3b3c7fff413086e3b58df52096f16b6891172`.
8
+
9
+ This is the coarse-to-fine paired GAN we retrained ourselves on QXS-SAROPT, and
10
+ it is the checkpoint behind the pix2pixHD row of the ReFlowSET comparison
11
+ table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `net_G.pth` | 729,795,505 | the global generator, and the only file inference needs |
18
+
19
+ `net_G.pth` is the **`global` generator** (`GlobalGenerator`):
20
+ `n_downsample_global` 4, `n_blocks_global` 9 residual blocks, `ngf = 64`,
21
+ instance normalisation, 3 in / 3 out. The run was configured with
22
+ `n_local_enhancers 1`, but at `netG=global` the local enhancer is never
23
+ constructed and is not in this file — this is the single-scale generator.
24
+
25
+ The run used `--label_nc 0 --no_instance`, i.e. the "label" input is a real
26
+ image (the SAR chip) rather than a segmentation map, and no instance-edge
27
+ channel is concatenated. The two discriminators (`num_D` 2) are not released.
28
+
29
+ ## Training budget we used
30
+
31
+ | | |
32
+ |---|---|
33
+ | **generator updates released** | **120,000** |
34
+ | batch size | 16 |
35
+ | epochs (`niter` + `niter_decay`) | 60 + 60 |
36
+ | iterations per epoch | 1,000 (16,001 training pairs; this loader floor-rounds, unlike the junyanz loaders which round up) |
37
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay over the second half |
38
+ | adversarial loss | LSGAN, multi-scale discriminator (`num_D` 2, `n_layers_D` 3) |
39
+ | feature-matching loss | λ_feat 10 |
40
+ | perceptual loss | VGG loss enabled |
41
+ | augmentation | horizontal flip |
42
+ | input | 256 px (`resize_and_crop`, `loadSize` 256, `fineSize` 256) |
43
+
44
+ The floor-vs-ceiling detail is not pedantry: pix2pixHD and SPADE drop the last
45
+ partial batch while the junyanz-family loaders do not, so the same "epochs"
46
+ figure yields slightly different update counts across the table. The update
47
+ counts above are what actually ran.
48
+
49
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
50
+
51
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
52
+ |---|---|---|---|---|
53
+ | 85.7 | 0.298 | 0.573 | 0.358 | 16.13 |
54
+
55
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
56
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
57
+
58
+ These are our own re-evaluation numbers, measured by us on the images this
59
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
60
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
61
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
62
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
63
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
64
+ convention gives a systematically different number and the two must never be
65
+ mixed, or compared against a paper that used the other one.
66
+
67
+ ## Load it and translate one SAR image
68
+
69
+ `GlobalGenerator` can be constructed directly, without the repository's option
70
+ machinery:
71
+
72
+ ```python
73
+ import numpy as np, torch, torch.nn as nn
74
+ from PIL import Image
75
+ from models.networks import GlobalGenerator # from the NVIDIA/pix2pixHD repo
76
+
77
+ G = GlobalGenerator(3, 3, 64, 4, 9, nn.InstanceNorm2d)
78
+ G.load_state_dict(torch.load('net_G.pth', map_location='cpu'))
79
+ G.eval()
80
+
81
+ sar = Image.open('sar.png').convert('RGB')
82
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
83
+
84
+ with torch.no_grad():
85
+ y = G(x)
86
+
87
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
88
+ Image.fromarray(eo).save('eo.png')
89
+ ```
90
+
91
+ Through the repository's own `test.py` the equivalent invocation is
92
+ `--label_nc 0 --no_instance --which_epoch latest`, and the output lands as
93
+ `<stem>_synthesized_image.png`.
94
+
95
+ ## Read before using this checkpoint
96
+
97
+ * **Upstream writes its test outputs as JPEG.** `util/visualizer.py` hard-codes
98
+ `'%s_%s.jpg'`. We changed that one line to `.png` before scoring, because
99
+ every other method in this benchmark writes PNG and JPEG re-compression would
100
+ have biased FID and LPIPS for this row alone. **If you clone upstream
101
+ unmodified and evaluate, your numbers will not match ours**, and the gap is an
102
+ artefact of the file format, not of the model.
103
+ * Three further changes we made to the repository are pure Python-3.12 /
104
+ modern-library compatibility and do not affect the weights:
105
+ `transforms.Scale` → `transforms.Resize`; `fractions.gcd` → `math.gcd` with
106
+ integer division; and a guarded `import scipy.misc` (needed only for
107
+ `--tf_log`, which we never pass).
108
+ * Normalisation is `[-1, 1]`.
109
+ * Passes the identity-collapse audit on both datasets.
110
+
111
+ ## Licence — stated factually
112
+
113
+ Upstream `NVIDIA/pix2pixHD` ships `LICENSE.txt`, reproduced verbatim here as
114
+ `licenses/LICENSE-p2phd.txt`. It contains two notices:
115
+
116
+ * `Copyright (C) 2019 NVIDIA Corporation. Ting-Chun Wang, Ming-Yu Liu,
117
+ Jun-Yan Zhu. BSD License. All rights reserved.` — two conditions and a
118
+ disclaimer;
119
+ * the bundled `pytorch-CycleGAN-and-pix2pix` notice,
120
+ `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park`, because pix2pixHD's code
121
+ borrows heavily from it.
122
+
123
+ **Redistribution of these weights is permitted** with both notices reproduced.
124
+ Despite NVIDIA's non-commercial licensing of some of its other releases, this
125
+ one carries **no non-commercial clause and no share-alike clause** — we checked
126
+ the file and the README, and the README has no licence section at all, so
127
+ `LICENSE.txt` is the entire grant.
128
+
129
+ Attribution: *trained with `NVIDIA/pix2pixHD`. Copyright (C) 2019 NVIDIA
130
+ Corporation — Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu. BSD License, all rights
131
+ reserved; includes the bundled pytorch-CycleGAN-and-pix2pix notice (Copyright
132
+ (c) 2017, Jun-Yan Zhu and Taesung Park). Full text in
133
+ `licenses/LICENSE-p2phd.txt`.*
134
+
135
+ ---
136
+
137
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
138
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
139
+ itself. Every comparison method in this repository was retrained by us on the
140
+ same splits at the same resolution and scored through one evaluation pipeline,
141
+ so the rows are directly comparable to each other — and, for the same reason,
142
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/pix2pix/README.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # pix2pix — QXS-SAROPT
2
+
3
+ **Image-to-Image Translation with Conditional Adversarial Networks**, Isola, Zhu,
4
+ Zhou and Efros, *CVPR 2017* ([arXiv:1611.07004](https://arxiv.org/abs/1611.07004)).
5
+ Upstream code:
6
+ [`junyanz/pytorch-CycleGAN-and-pix2pix`](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
7
+ at commit `2a7afba2895d52556dd5dfe07e8555ef657ced6f`.
8
+
9
+ This is the paired conditional GAN we retrained ourselves on QXS-SAROPT, and it
10
+ is the checkpoint behind the pix2pix row of the ReFlowSET comparison table.
11
+
12
+ ## What is in this folder
13
+
14
+ | file | bytes | what it is |
15
+ |---|---|---|
16
+ | `net_G.pth` | 217,728,293 | the generator, and the only file inference needs |
17
+
18
+ `net_G.pth` is a **`unet_256` generator**: a U-Net with batch normalisation,
19
+ `ngf = 64`, 3 input channels and 3 output channels, saved from a single-GPU run
20
+ so the state-dict keys are bare (no `module.` prefix). It maps **SAR → EO**:
21
+ the training set was built as side-by-side `A|B` tiles with SAR on the left, and
22
+ the run used `--direction AtoB`.
23
+
24
+ The discriminator is not part of this release — upstream's `test.py` constructs
25
+ only the generator, so `net_G.pth` is sufficient to reproduce every image we
26
+ scored.
27
+
28
+ ## Training budget we used
29
+
30
+ | | |
31
+ |---|---|
32
+ | **generator updates released** | **120,120** |
33
+ | batch size | 16 |
34
+ | epochs (constant LR + linear decay) | 60 + 60 |
35
+ | iterations per epoch | 1,001 (16,001 training pairs, the loader does not drop the last batch) |
36
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
37
+ | adversarial loss | vanilla GAN; discriminator `basic` (70×70 PatchGAN, `n_layers_D` 3) |
38
+ | reconstruction loss | L1, weight 100 |
39
+ | augmentation | horizontal flip |
40
+ | input | 256 px, no resize (the chips are already 256 px) |
41
+
42
+ We quote budgets in **optimizer updates**, never in epochs: the two datasets
43
+ differ by more than 10× in size, so "200 epochs" means something completely
44
+ different on each and would silently favour one of them.
45
+
46
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
47
+
48
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
49
+ |---|---|---|---|---|
50
+ | 174.6 | 0.373 | 0.665 | 0.203 | 12.33 |
51
+
52
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
53
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
54
+
55
+ These are our own re-evaluation numbers, measured by us on the images this
56
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
57
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
58
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
59
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
60
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
61
+ convention gives a systematically different number and the two must never be
62
+ mixed, or compared against a paper that used the other one.
63
+
64
+ ## Load it and translate one SAR image
65
+
66
+ The snippet uses the upstream repository's own `define_G`, so clone
67
+ `junyanz/pytorch-CycleGAN-and-pix2pix` at the commit above and run this from its
68
+ root. No patch is needed for this method — our only change to that repository was
69
+ an import fix in the dataset-preparation helper, which does not affect the model.
70
+
71
+ ```python
72
+ import numpy as np, torch
73
+ from PIL import Image
74
+ from models.networks import define_G # from the junyanz repo
75
+
76
+ G = define_G(3, 3, 64, 'unet_256', 'batch', use_dropout=False,
77
+ init_type='normal', init_gain=0.02, gpu_ids=[])
78
+ sd = torch.load('net_G.pth', map_location='cpu')
79
+ if hasattr(sd, '_metadata'):
80
+ del sd._metadata # upstream strips this too
81
+ G.load_state_dict(sd)
82
+ G.eval()
83
+
84
+ # SAR input -> RGB -> [-1, 1]. QXS-SAROPT SAR is single-channel; .convert('RGB')
85
+ # replicates it to three channels, which is what training saw.
86
+ sar = Image.open('sar.png').convert('RGB')
87
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
88
+
89
+ with torch.no_grad():
90
+ y = G(x)
91
+
92
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
93
+ Image.fromarray(eo).save('eo.png')
94
+ ```
95
+
96
+ The image this produces is `<stem>_fake_B.png` in upstream's own `test.py`
97
+ output tree, which is exactly what we scored.
98
+
99
+ ## Read before using this checkpoint
100
+
101
+ * **`define_G` must be called with `gpu_ids=[]`.** Our checkpoints were saved
102
+ from single-GPU runs and have bare keys; building the network with a non-empty
103
+ `gpu_ids` wraps it in `DataParallel` and the load then fails on a `module.`
104
+ prefix mismatch.
105
+ * **The normalisation is `[-1, 1]`, not `[0, 1]`.** Feeding `[0, 1]` produces a
106
+ plausible-looking but systematically wrong image.
107
+ * This row is a **paired** method and was trained on aligned SAR/EO pairs. It
108
+ passes the identity-collapse audit on both datasets.
109
+
110
+ ## Licence — stated factually
111
+
112
+ Upstream `junyanz/pytorch-CycleGAN-and-pix2pix` ships a LICENSE file, reproduced
113
+ verbatim in this repository as `licenses/LICENSE-pix2pix.txt`. It is **not a
114
+ single SPDX licence** but three stacked notices, and the whole file is what
115
+ travels with these weights:
116
+
117
+ * CycleGAN — `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park`, two conditions
118
+ and a disclaimer (BSD-2-Clause in form);
119
+ * pix2pix — `Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu`, two conditions;
120
+ * DCGAN — `Copyright (c) 2015, Facebook, Inc.`, three conditions including a
121
+ no-endorsement clause (BSD-3-Clause).
122
+
123
+ **Redistribution of these weights is permitted** with that notice reproduced.
124
+ There is no non-commercial and no share-alike clause. The DCGAN section's
125
+ no-endorsement clause means Facebook's name must not be used to promote this
126
+ release.
127
+
128
+ Attribution: *trained with `junyanz/pytorch-CycleGAN-and-pix2pix`. Copyright (c)
129
+ 2017, Jun-Yan Zhu and Taesung Park (CycleGAN); Copyright (c) 2016, Phillip Isola
130
+ and Jun-Yan Zhu (pix2pix); Copyright (c) 2015, Facebook, Inc. (DCGAN). All rights
131
+ reserved. Redistributed under the BSD terms reproduced in
132
+ `licenses/LICENSE-pix2pix.txt`.*
133
+
134
+ ---
135
+
136
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
137
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
138
+ itself. Every comparison method in this repository was retrained by us on the
139
+ same splits at the same resolution and scored through one evaluation pipeline,
140
+ so the rows are directly comparable to each other — and, for the same reason,
141
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/resshift/README.md ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ResShift — QXS-SAROPT
2
+
3
+ **ResShift: Efficient Diffusion Model for Image Super-Resolution by Residual
4
+ Shifting**, Yue, Wang and Loy, *NeurIPS 2023*
5
+ ([arXiv:2307.12348](https://arxiv.org/abs/2307.12348)). Upstream code:
6
+ [`zsyOAOA/ResShift`](https://github.com/zsyOAOA/ResShift) at commit
7
+ `bb03b7d21614cace01787e097c8a6ab6b945227d`.
8
+
9
+ This is the residual-shifting latent diffusion model we retrained ourselves on
10
+ QXS-SAROPT, run as a **same-size** mapping (`sf = 1`: SAR in, EO out, no
11
+ upscaling), and it is the checkpoint behind the ResShift row of the ReFlowSET
12
+ comparison table.
13
+
14
+ ## What is in this folder
15
+
16
+ | file | bytes | what it is |
17
+ |---|---|---|
18
+ | `ema_model.pth` | 478,392,663 | EMA weights of the denoising UNet at iteration 50,000 |
19
+
20
+ `ema_model.pth` is the EMA (rate 0.999) of a **`models.unet.UNetModelSwin`**:
21
+ latent resolution 64, `model_channels` 160, `channel_mult` [1, 2, 2, 4],
22
+ `num_res_blocks` [2, 2, 2, 2], attention at [64, 32, 16, 8], 32 channels per
23
+ head, Swin blocks of depth 2 with `swin_embed_dim` 192 and window size 8,
24
+ `cond_lq = True`, `lq_size = 256`.
25
+
26
+ **This is a latent-space model and this file is not sufficient to run it.** It
27
+ additionally needs the external CompVis **vq-f4** autoencoder
28
+ (`ldm.models.autoencoder.VQModelTorch`, 221,364,711 bytes), which is **not**
29
+ redistributed here.
30
+
31
+ ## Training budget we used
32
+
33
+ | | |
34
+ |---|---|
35
+ | **generator updates released** | **50,000** |
36
+ | batch size | 16 (micro-batch 8, i.e. 2 gradient-accumulation steps per update) |
37
+ | optimizer | Adam, lr 5e-5 → 2e-5 cosine, 2,000-step warmup, no weight decay |
38
+ | EMA | rate 0.999 |
39
+ | diffusion | **4 steps**, exponential noise schedule (power 0.3), `etas_end` 0.99, `min_noise_level` 0.2, κ = 2.0, `predict_type` xstart, `sf` 1 |
40
+ | loss weights | latent MSE 1.0, pixel LPIPS (AlexNet) 4.0, pixel MSE 0.0 |
41
+ | precision | AMP on |
42
+ | training crops | random 256 px, normalised with mean 0.5 / std 0.5 |
43
+ | seed | 123456 |
44
+
45
+ ResShift is a **4-step** sampler by construction; that is the method, not a
46
+ shortcut we took.
47
+
48
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
49
+
50
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
51
+ |---|---|---|---|---|
52
+ | 140.2 | 0.334 | 0.607 | 0.217 | 14.20 |
53
+
54
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
55
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
56
+
57
+ These are our own re-evaluation numbers, measured by us on the images this
58
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
59
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
60
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
61
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
62
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
63
+ convention gives a systematically different number and the two must never be
64
+ mixed, or compared against a paper that used the other one.
65
+
66
+ ## Load it and translate one SAR image
67
+
68
+ Upstream's `inference_resshift.py` is a super-resolution CLI keyed to the
69
+ authors' own released tasks and does not accept a SAR→EO configuration. We wrote
70
+ a small driver against the repository's `ResShiftSampler`; the equivalent is:
71
+
72
+ ```python
73
+ from omegaconf import OmegaConf
74
+ from sampler import ResShiftSampler # from the zsyOAOA/ResShift repo
75
+
76
+ cfg = OmegaConf.load('<your>_sar2eo_256.yaml') # sf 1, steps 4, predict_type xstart
77
+ cfg.model.ckpt_path = 'ema_model.pth'
78
+ cfg.autoencoder.ckpt_path = '<path to>/autoencoder_vq_f4.pth' # NOT shipped here
79
+
80
+ sampler = ResShiftSampler(cfg, sf=1, chop_size=256, chop_stride=256,
81
+ chop_bs=1, use_amp=True, seed=12345, padding_offset=None)
82
+ sampler.inference('<dir of SAR images>', '<output dir>', bs=16, noise_repeat=False)
83
+ ```
84
+
85
+ At 256 px no tiling happens. Outputs keep the input file stems.
86
+
87
+ ## Read before using this checkpoint
88
+
89
+ * **The external vq-f4 autoencoder is mandatory and is not in this repository.**
90
+ Without it the model cannot be constructed, let alone run. It is CompVis
91
+ latent-diffusion material under its own (MIT) licence.
92
+ * **Upstream's own inference CLI will not drive these weights.** It is written
93
+ around the authors' released super-resolution tasks. Use the sampler directly,
94
+ as above.
95
+ * **xFormers must be disabled.** On the accelerator we trained on, the installed
96
+ xFormers build has no usable kernel for these shapes (attention head dimension
97
+ 512 exceeds its limit, and fp32 is unsupported), so we set
98
+ `XFORMERS_IS_AVAILBLE = False` in three files
99
+ (`ldm/modules/attention.py`, `ldm/modules/diffusionmodules/model.py`,
100
+ `models/unet.py`). If your hardware differs, this is a performance switch, not
101
+ a correctness one.
102
+ * **One numerical guard in the image logger.** fp16 diffused latents decoded
103
+ through the fp32 VQGAN produce NaN/Inf at early iterations and crashed training
104
+ *inside the logging path*. We added a `nan_to_num` + `clamp` before
105
+ `make_grid`. It is logging-only; the loss path is untouched. (A separate
106
+ `nan_to_num` on the LPIPS loss is upstream's, not ours.)
107
+ * **There is no auto-resume.** A crash mid-run needs a manual relaunch pointing
108
+ at the last saved checkpoint.
109
+ * Passes the identity-collapse audit on both datasets.
110
+
111
+ ## Licence — stated factually ⚠ non-commercial
112
+
113
+ Upstream `zsyOAOA/ResShift` ships a LICENSE file, reproduced verbatim here as
114
+ `licenses/LICENSE-resshift.txt`: the **S-Lab License 1.0**,
115
+ `Copyright 2022 S-Lab`.
116
+
117
+ **This licence is non-commercial.** Its opening clause reads *"Redistribution
118
+ and use **for non-commercial purpose** in source and binary forms, with or
119
+ without modification, are permitted provided that the following conditions are
120
+ met"*, and it adds that commercial use requires contacting the contributors. The
121
+ README repeats it: *"This project is licensed under NTU S-Lab License 1.0.
122
+ Redistribution and use should follow this license."*
123
+
124
+ So: **redistribution of these weights is permitted for non-commercial purposes
125
+ only**, in binary form, with the copyright notice, the conditions and the
126
+ disclaimer reproduced — which is what `licenses/LICENSE-resshift.txt` is for.
127
+ Clause 3 is a **no-endorsement** clause: neither S-Lab's name nor its
128
+ contributors' names may be used to endorse or promote this release.
129
+
130
+ Attribution: *trained with `zsyOAOA/ResShift`, Copyright 2022 S-Lab, **S-Lab
131
+ License 1.0 — redistribution and use for non-commercial purposes only**; for
132
+ commercial use, contact the contributors. Requires the CompVis vq-f4
133
+ autoencoder (MIT), not redistributed here. Neither S-Lab's name nor its
134
+ contributors' names may be used to endorse this release. Full text in
135
+ `licenses/LICENSE-resshift.txt`.*
136
+
137
+ ---
138
+
139
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
140
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
141
+ itself. Every comparison method in this repository was retrained by us on the
142
+ same splits at the same resolution and scored through one evaluation pipeline,
143
+ so the rows are directly comparable to each other — and, for the same reason,
144
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/sd21ft/README.md ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SD2.1 fine-tune only — QXS-SAROPT
2
+
3
+ The **"SD2.1 fine-tune only"** ablation row of the ReFlowSET comparison table:
4
+ Stable Diffusion 2.1-base fine-tuned for SAR-to-EO translation with a plain MSE
5
+ epsilon loss and **no confidence head**.
6
+
7
+ > **This is not a stock SD 2.1 UNet and it will not load into a stock SD 2.1
8
+ > pipeline.** It is C-DiffSET **stage 1**: the input convolution is widened to
9
+ > 8 channels so the SAR latent can be concatenated to the noisy EO latent. Only
10
+ > the output side is unmodified.
11
+
12
+ Venue: there is no separate paper for this row. The backbone is Stable
13
+ Diffusion 2.1-base (latent diffusion, Rombach et al., CVPR 2022); the trainer
14
+ is the stage-1 trainer of
15
+ [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET)
16
+ (*IEEE TCSVT* 2026, [doi:10.1109/TCSVT.2026.3701447](https://doi.org/10.1109/TCSVT.2026.3701447)).
17
+
18
+ ## What is in this folder
19
+
20
+ A single `UNet2DConditionModel` (865,922,244 parameters, fp32, 686 tensors):
21
+
22
+ * **`conv_in` 4 -> 8 channels**, exactly as in the C-DiffSET folder — the input
23
+ is `torch.cat([sar_latent, noisy_eo_latent], dim=1)`, **SAR first**. In this
24
+ trained checkpoint the two halves have diverged (mean |w| 0.01443 on channels
25
+ 0-3 vs 0.02727 on channels 4-7).
26
+ * **`conv_out` unchanged at 4 channels.** No variance head, no confidence map.
27
+ The stage-1 trainer explicitly overrides the channel surgery to a no-op, and
28
+ the whole 4-channel prediction goes to the scheduler.
29
+
30
+ That 5-vs-4 output channel count is the only architectural difference from the
31
+ `cdiffset` folder in this repository; the parameter counts differ by exactly
32
+ 2,881 (one 320x3x3 filter plus one bias).
33
+
34
+ ## Training budget we used
35
+
36
+ | | |
37
+ |---|---|
38
+ | optimizer updates released | **40,000** (fixed-step snapshot `checkpoint-40000`) |
39
+ | updates the run was configured for | 50,000 |
40
+ | initialised from | the SD 2.1-base UNet with the duplicated-and-halved 8-channel `conv_in` |
41
+ | batch size / resolution | 64 @ 256 px |
42
+ | epochs the 40,000 updates work out to | ~160 (16,001 training pairs, 250 updates/epoch) |
43
+ | optimizer | AdamW, lr 3e-5, weight decay 0.01, cosine schedule, 100-step warmup |
44
+ | precision | fp32 (no mixed precision) |
45
+ | seed | 2024 |
46
+ | loss | plain MSE on the epsilon prediction |
47
+ | prompt | the fixed string `"electro-optical image"` |
48
+ | augmentation | horizontal flip, vertical flip, 90-degree rotation |
49
+ | test-time sampler | **DDIM, 50 steps**, fp32, one image at a time |
50
+
51
+ Same two caveats as the C-DiffSET folder: the snapshot is taken at update 40,000
52
+ of a cosine schedule laid out for 50,000, so the learning rate had not finished
53
+ decaying; and it is a fixed-step snapshot, not the repository's
54
+ validation-selected `best/`. For this stage the fixed-step choice is also the
55
+ better one on the merits — validation LPIPS bottoms out near update 41,000 and
56
+ then degrades toward 50,000.
57
+
58
+ Data: 256 px, no crop and no resize (the chips are already 256 px). SAR chips come from the `sar_256_oc_0.2` directory of QXS-SAROPT; the EO target is the same stem under `opt_256_oc_0.2`.
59
+
60
+ ## Base model
61
+
62
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
63
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
64
+ available on the Hub, our training and every snippet below load the community
65
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
66
+ That is what we actually trained against — verified in the training
67
+ configuration files, the launch scripts and the training logs, none of which
68
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
69
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
70
+ expected to work but has not been verified by us.
71
+
72
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
73
+
74
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
75
+ |---|---|---|---|---|---|
76
+ | 19.1 | 0.0042 | 0.257 | 0.561 | 15.40 | 0.348 |
77
+
78
+ These are our own re-evaluation numbers, not numbers copied from any paper.
79
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
80
+ convention); the alternative [0, 1] convention gives a systematically different
81
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
82
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
83
+
84
+ ## Load it and translate one SAR PNG
85
+
86
+ ```python
87
+ import numpy as np
88
+ import torch
89
+ from PIL import Image
90
+ from diffusers import UNet2DConditionModel, AutoencoderKL, DDIMScheduler
91
+ from transformers import CLIPTextModel, CLIPTokenizer
92
+
93
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
94
+ BASE = "Manojb/stable-diffusion-2-1-base" # frozen VAE / text encoder / scheduler
95
+ DEV, DT = "cuda", torch.float32
96
+
97
+ unet = UNet2DConditionModel.from_pretrained(
98
+ REPO, subfolder="baselines/qxs-saropt/sd21ft", torch_dtype=DT).to(DEV).eval()
99
+ vae = AutoencoderKL.from_pretrained(BASE, subfolder="vae", torch_dtype=DT).to(DEV).eval()
100
+ tok = CLIPTokenizer.from_pretrained(BASE, subfolder="tokenizer")
101
+ txt = CLIPTextModel.from_pretrained(BASE, subfolder="text_encoder", torch_dtype=DT).to(DEV).eval()
102
+ sch = DDIMScheduler.from_pretrained(BASE, subfolder="scheduler")
103
+ sch.set_timesteps(50, device=DEV)
104
+
105
+ # The fixed training prompt, tokenized exactly as in training (no padding).
106
+ ids = tok("electro-optical image", padding="do_not_pad",
107
+ max_length=tok.model_max_length, truncation=True,
108
+ return_tensors="pt").input_ids.to(DEV)
109
+ with torch.no_grad():
110
+ embed = txt(ids)[0]
111
+
112
+ # SAR input: 256x256 PNG -> RGB -> [-1, 1]. Do not resize; crop if needed.
113
+ sar = Image.open("sar.png").convert("RGB")
114
+ x = torch.from_numpy(np.array(sar)).permute(2, 0, 1)[None]
115
+ x = (x.to(DEV, DT) / 255.0) * 2 - 1
116
+
117
+ with torch.no_grad():
118
+ sar_lat = vae.encode(x).latent_dist.mean * vae.config.scaling_factor # .mean, not .sample()
119
+ eo_lat = torch.randn_like(sar_lat)
120
+ for t in sch.timesteps:
121
+ out = unet(torch.cat([sar_lat, eo_lat], 1), # SAR first, then the noisy EO latent
122
+ t, encoder_hidden_states=embed).sample
123
+ eo_lat = sch.step(out, t, eo_lat).prev_sample # all 4 channels; no variance head
124
+ eo = vae.decode(eo_lat / vae.config.scaling_factor).sample
125
+ eo = (eo * 0.5 + 0.5).clamp(0, 1)[0].cpu()
126
+
127
+ Image.fromarray((eo.permute(1, 2, 0).numpy() * 255).round().astype(np.uint8)).save("eo.png")
128
+ ```
129
+
130
+ The two things that silently produce a plausible-but-wrong image if you get
131
+ them backwards: the concatenation order is **SAR first**, and the VAE must be
132
+ read as `latent_dist.mean`, not `.sample()`.
133
+
134
+ ## Licence status — stated factually
135
+
136
+ * **Training code:** MIT, `Copyright (c) 2026 KAIST VICLab` (the stage-1
137
+ trainer of [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET)).
138
+ The row is named after Stable Diffusion, but no stabilityai code was used and
139
+ no stabilityai repository was loaded.
140
+ * **These weights:** a fine-tune of Stable Diffusion 2.1-base, hence a
141
+ *Derivative of the Model* under the **CreativeML Open RAIL++-M** licence
142
+ (`openrail++`). The **Attachment A use restrictions travel with these
143
+ weights** and must be passed on to anyone you redistribute to. Full text,
144
+ including Attachment A, ships in this repository as `LICENSE-WEIGHTS.md`.
145
+
146
+ Attribution: *fine-tuned from Stable Diffusion 2.1-base (obtained as
147
+ `Manojb/stable-diffusion-2-1-base`, a re-upload of
148
+ `stabilityai/stable-diffusion-2-1-base`) using the stage-1 trainer of
149
+ `KAIST-VICLab/C-DiffSET`.*
150
+
151
+ ---
152
+
153
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
154
+ checkpoint retrained by us on QXS-SAROPT; it is not ReFlowSET itself. Every
155
+ comparison method in the release was retrained by us on the same splits at the
156
+ same resolution, so the rows are directly comparable to each other.
baselines/qxs-saropt/sd21ft/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "UNet2DConditionModel",
3
+ "_diffusers_version": "0.10.0.dev0",
4
+ "act_fn": "silu",
5
+ "attention_head_dim": [
6
+ 5,
7
+ 10,
8
+ 20,
9
+ 20
10
+ ],
11
+ "block_out_channels": [
12
+ 320,
13
+ 640,
14
+ 1280,
15
+ 1280
16
+ ],
17
+ "center_input_sample": false,
18
+ "cross_attention_dim": 1024,
19
+ "down_block_types": [
20
+ "CrossAttnDownBlock2D",
21
+ "CrossAttnDownBlock2D",
22
+ "CrossAttnDownBlock2D",
23
+ "DownBlock2D"
24
+ ],
25
+ "downsample_padding": 1,
26
+ "dual_cross_attention": false,
27
+ "flip_sin_to_cos": true,
28
+ "freq_shift": 0,
29
+ "in_channels": 8,
30
+ "layers_per_block": 2,
31
+ "mid_block_scale_factor": 1,
32
+ "norm_eps": 1e-05,
33
+ "norm_num_groups": 32,
34
+ "num_class_embeds": null,
35
+ "only_cross_attention": false,
36
+ "out_channels": 4,
37
+ "sample_size": 64,
38
+ "up_block_types": [
39
+ "UpBlock2D",
40
+ "CrossAttnUpBlock2D",
41
+ "CrossAttnUpBlock2D",
42
+ "CrossAttnUpBlock2D"
43
+ ],
44
+ "use_linear_projection": true
45
+ }
baselines/qxs-saropt/spade/README.md ADDED
@@ -0,0 +1,176 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPADE — QXS-SAROPT
2
+
3
+ **Semantic Image Synthesis with Spatially-Adaptive Normalization**, Park, Liu,
4
+ Wang and Zhu, *CVPR 2019* ([arXiv:1903.07291](https://arxiv.org/abs/1903.07291)).
5
+ Upstream code: [`NVlabs/SPADE`](https://github.com/NVlabs/SPADE) at commit
6
+ `fecacc920c1367a038995c45a39c15f6521ca64f`.
7
+
8
+ This is the spatially-adaptive-normalisation generator we retrained ourselves on
9
+ QXS-SAROPT, and it is the checkpoint behind the SPADE row of the ReFlowSET
10
+ comparison table.
11
+
12
+ **This checkpoint will not load into a clean clone of `NVlabs/SPADE`.** Read the
13
+ next two sections before trying.
14
+
15
+ ## What is in this folder
16
+
17
+ | file | bytes | what it is |
18
+ |---|---|---|
19
+ | `net_G.pth` | 368,775,707 | the SPADE generator, and the only file inference needs |
20
+
21
+ `net_G.pth` is a **`SPADEGenerator`**: `ngf = 64`,
22
+ `norm_G = spectralspadesyncbatch3x3`, `num_upsampling_layers = normal`,
23
+ `use_vae = False` (no style encoder — the generator is deterministic given the
24
+ input), and **`semantic_nc = 3`**.
25
+
26
+ That `semantic_nc = 3` is the whole story of this cell. SPADE is a *semantic
27
+ image synthesis* model: upstream one-hot-encodes its "label" input into
28
+ `label_nc` channels. A SAR chip is a real image, not a segmentation map, so we
29
+ run with `--label_nc 0 --no_instance`, which upstream treats as an error
30
+ condition. Stock SPADE therefore never constructs a generator with a 3-channel
31
+ spatially-adaptive input and cannot load this file. The two discriminators
32
+ (`num_D` 2) are not released.
33
+
34
+ ## Training budget we used
35
+
36
+ | | |
37
+ |---|---|
38
+ | **generator updates released** | **120,000** |
39
+ | batch size | 16 |
40
+ | epochs (`niter` + `niter_decay`) | 60 + 60 |
41
+ | iterations per epoch | 1,000 (16,001 training pairs; this loader drops the last partial batch) |
42
+ | optimizer | Adam, lr 2e-4 with TTUR (`no_TTUR` off, so G runs at lr/2 and D at 2·lr), β₁ 0.0, β₂ 0.9 |
43
+ | adversarial loss | hinge GAN, multi-scale discriminator (`num_D` 2, `n_layers_D` 4) |
44
+ | feature-matching loss | λ_feat 10 |
45
+ | perceptual loss | VGG loss, λ_vgg 10 |
46
+ | initialisation | Xavier |
47
+ | augmentation | horizontal flip |
48
+ | input | 256 px (`resize_and_crop`, `load_size` 256, `crop_size` 256) |
49
+
50
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
51
+
52
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
53
+ |---|---|---|---|---|
54
+ | 90.7 | 0.292 | 0.599 | 0.320 | 14.53 |
55
+
56
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
57
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
58
+
59
+ These are our own re-evaluation numbers, measured by us on the images this
60
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
61
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
62
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
63
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
64
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
65
+ convention gives a systematically different number and the two must never be
66
+ mixed, or compared against a paper that used the other one.
67
+
68
+ ## Load it and translate one SAR image
69
+
70
+ **You need the image-to-image adaptation described below**, applied to the
71
+ upstream repository, before this will run. With that in place, the generator can
72
+ be constructed directly:
73
+
74
+ ```python
75
+ import numpy as np, torch
76
+ from argparse import Namespace
77
+ from PIL import Image
78
+ from models.networks.generator import SPADEGenerator # from the NVlabs/SPADE repo
79
+
80
+ opt = Namespace(semantic_nc=3, label_nc=0, no_instance=True, ngf=64,
81
+ num_upsampling_layers='normal',
82
+ norm_G='spectralspadesyncbatch3x3',
83
+ crop_size=256, aspect_ratio=1.0, use_vae=False)
84
+ G = SPADEGenerator(opt)
85
+ G.load_state_dict(torch.load('net_G.pth', map_location='cpu'))
86
+ G.eval()
87
+
88
+ sar = Image.open('sar.png').convert('RGB')
89
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
90
+
91
+ with torch.no_grad():
92
+ y = G(seg=x, z=None) # the SAR image goes in the 'seg' slot
93
+
94
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
95
+ Image.fromarray(eo).save('eo.png')
96
+ ```
97
+
98
+ Through the repository's own `test.py`, the invocation is
99
+ `--dataset_mode custom --label_nc 0 --no_instance --no_pairing_check` with the
100
+ SAR directory as `--label_dir`. Note that the paired loader also demands an
101
+ `--image_dir` even at test time, so a SAR-only run needs either a dummy EO
102
+ directory or the direct generator call above.
103
+
104
+ ## Read before using this checkpoint
105
+
106
+ **The five changes that make this cell exist.** These are a functional
107
+ adaptation, not compatibility fixes, and a reproducer must apply all of them:
108
+
109
+ 1. `data/pix2pix_dataset.py` — when `label_nc == 0`, load the "label" through
110
+ the *image* transform (`.convert('RGB')`, normalised to `[-1, 1]`) instead of
111
+ nearest-neighbour resampling, ×255 and the `==255 → label_nc` remap. The SAR
112
+ PNG is a real image.
113
+ 2. `models/pix2pix_model.py` — an early return in `preprocess_input()`: when
114
+ `label_nc == 0`, skip the `.long()` cast and the `scatter_` one-hot encoding
115
+ and pass the float image straight through as `input_semantics`. One-hot
116
+ encoding a normalised float image is undefined.
117
+ 3. `options/base_options.py` — when `label_nc == 0`, force
118
+ `opt.semantic_nc = 3 + (0 if no_instance else 1)`. Upstream computes
119
+ `semantic_nc = label_nc + …`, i.e. 0 or 1, and would build a generator with a
120
+ one-channel spatially-adaptive input.
121
+ 4. `models/networks/discriminator.py` — the discriminator input width becomes
122
+ `(label_nc if label_nc > 0 else 3) + output_nc`, for the same reason on the
123
+ discriminator side.
124
+ 5. `util/visualizer.py` — `tensor2label` is given `n_label=0` when
125
+ `label_nc == 0` so it falls through to `tensor2im` instead of colour-mapping
126
+ a real image as a label map; and `import scipy.misc` is guarded, since it was
127
+ removed in SciPy ≥ 1.12.
128
+
129
+ Changes 1–4 change what the network *is*. **Without them this file does not
130
+ load.**
131
+
132
+ * Normalisation is `[-1, 1]`.
133
+ * `norm_G` uses synchronised batch normalisation; on a single device this
134
+ behaves as ordinary batch norm, and `.eval()` is required.
135
+ * Passes the identity-collapse audit on both datasets.
136
+
137
+ ## Licence — stated factually ⚠ non-commercial and share-alike
138
+
139
+ Upstream `NVlabs/SPADE` ships `LICENSE.md`, reproduced verbatim here as
140
+ `licenses/LICENSE-spade.txt`. It is the full legal code of
141
+ **Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International
142
+ (CC BY-NC-SA 4.0)**. The copyright holder is named in the README rather than in
143
+ the licence file: *"Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
144
+ … The code is released for academic research use only. For commercial use or
145
+ business inquiries, please contact researchinquiries@nvidia.com."*
146
+
147
+ **This is the most restrictive licence in this repository, and two of its terms
148
+ travel with this checkpoint:**
149
+
150
+ * **NonCommercial.** You may not use this checkpoint for commercial advantage or
151
+ monetary compensation.
152
+ * **ShareAlike.** If a trained checkpoint is Adapted Material — and we treat it
153
+ as such — then any licence you apply when you pass it on must be a Creative
154
+ Commons licence with the same elements, this version or later, or a BY-NC-SA
155
+ compatible licence. **This cell is therefore distributed under CC BY-NC-SA
156
+ 4.0**; it cannot sit under a permissive repository-wide weight licence, and
157
+ neither can anything you derive from it.
158
+ * The licence also states that patent and trademark rights are not licensed.
159
+ * NVIDIA's academic-use-only statement in the README is an additional assertion
160
+ beyond the CC text.
161
+
162
+ Attribution: *trained with `NVlabs/SPADE`. Copyright (C) 2019 NVIDIA
163
+ Corporation, all rights reserved. Licensed under CC BY-NC-SA 4.0; NVIDIA
164
+ releases the code for academic research use only (commercial enquiries:
165
+ researchinquiries@nvidia.com). This checkpoint is therefore distributed under
166
+ CC BY-NC-SA 4.0 — non-commercial, share-alike. Full text in
167
+ `licenses/LICENSE-spade.txt`.*
168
+
169
+ ---
170
+
171
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
172
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
173
+ itself. Every comparison method in this repository was retrained by us on the
174
+ same splits at the same resolution and scored through one evaluation pipeline,
175
+ so the rows are directly comparable to each other — and, for the same reason,
176
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/qxs-saropt/stegogan/README.md ADDED
@@ -0,0 +1,200 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # StegoGAN — QXS-SAROPT
2
+
3
+ **StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image
4
+ Translation**, Wu, Chen, Mermet, Hurni, Schindler, Gonthier and Landrieu,
5
+ *CVPR 2024*. Upstream code:
6
+ [`sian-wusidi/StegoGAN`](https://github.com/sian-wusidi/StegoGAN) (the official
7
+ CVPR 2024 repository) at commit `cad61997c0f82793444f60f81298142b80cdf3c1`.
8
+
9
+ This is the mismatch-aware unpaired GAN we retrained ourselves on QXS-SAROPT,
10
+ and it is the checkpoint behind the StegoGAN row of the ReFlowSET comparison
11
+ table. **Two things on this page are load-bearing: there is an oracle output you
12
+ must not use, and this cell fails our input-copy audit.**
13
+
14
+ We applied **no patches** to this repository. It is stock upstream at that
15
+ commit.
16
+
17
+ ## What is in this folder
18
+
19
+ | file | bytes | what it is |
20
+ |---|---|---|
21
+ | `net_G_A.pth` | 50,254,749 | **SAR → EO** (`resnet_9blocks_maskv1`) — the generator that produces the reported row |
22
+ | `net_G_B.pth` | 52,615,525 | EO → SAR (`resnet_9blocks_maskv3`), which also returns the mismatch mask; used to close the cycle in training, and **not needed for a prediction** |
23
+
24
+ Both generators are ResNet-style with instance normalisation, `ngf = 64`, built
25
+ with `--resnet_layer 8 --fusionblock`. **Those two are architecture flags, not
26
+ training options**: build the network without them and the state dict will not
27
+ load.
28
+
29
+ `net_G_A` takes an **optional second argument**. That option is the whole reason
30
+ this card has a warning section — see below.
31
+
32
+ ## Training budget we used
33
+
34
+ | | |
35
+ |---|---|
36
+ | **data iterations released** | **120,030** |
37
+ | generator optimizer steps | 240,060 — see the note below |
38
+ | batch size | 4 |
39
+ | epochs (constant LR + linear decay) | 15 + 15 |
40
+ | iterations per epoch | 4,001 (16,001 training images, last batch not dropped) |
41
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
42
+ | adversarial loss | LSGAN; discriminator `basic` (70×70 PatchGAN) |
43
+ | cycle losses | λ_A 10, λ_B 10 |
44
+ | identity loss | λ_identity 0.5 |
45
+ | consistency / regularisation | λ_consistency 1.0, λ_reg 0.3 |
46
+ | augmentation | horizontal flip |
47
+ | input | 256 px, no resize |
48
+
49
+ **Why two numbers.** StegoGAN's training step calls `optimizer_G.step()`
50
+ **twice** off a single backward pass, so its optimizer-step count is exactly
51
+ double its data-iteration count. We quote StegoGAN budgets in *data iterations*
52
+ throughout this benchmark and note the 2× explicitly. Do not re-derive one cell
53
+ the other way — the family stops being comparable with itself.
54
+
55
+ ## Measured on the QXS-SAROPT test set (n = 3,999, 256 px)
56
+
57
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
58
+ |---|---|---|---|---|
59
+ | 106.8 | 0.384 | 0.658 | 0.254 | 12.96 |
60
+
61
+ Evaluated on the official split's 3,999 test chips at their native 256 px.
62
+ No subsampling: every metric on this page is measured over all 3,999 pairs.
63
+
64
+ These are our own re-evaluation numbers, measured by us on the images this
65
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
66
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
67
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
68
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
69
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
70
+ convention gives a systematically different number and the two must never be
71
+ mixed, or compared against a paper that used the other one.
72
+
73
+ ## Load it and translate one SAR image
74
+
75
+ Clone `sian-wusidi/StegoGAN` at the commit above and run this from its root.
76
+ Check `define_G`'s signature in that repository first: StegoGAN adds
77
+ `resnet_layer` and `fusionblock` keyword arguments that the junyanz original does
78
+ not have.
79
+
80
+ ```python
81
+ import numpy as np, torch
82
+ from PIL import Image
83
+ from models.networks import define_G # from the StegoGAN repo
84
+
85
+ G_A = define_G(3, 3, 64, 'resnet_9blocks_maskv1', 'instance', use_dropout=False,
86
+ init_type='normal', init_gain=0.02, gpu_ids=[],
87
+ resnet_layer=8, fusionblock=True)
88
+ G_A.load_state_dict(torch.load('net_G_A.pth', map_location='cpu'))
89
+ G_A.eval()
90
+
91
+ sar = Image.open('sar.png').convert('RGB')
92
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
93
+
94
+ with torch.no_grad():
95
+ fake_B_clean = G_A(x) # ONE argument. Passing a second makes it an oracle.
96
+
97
+ eo = ((fake_B_clean[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
98
+ Image.fromarray(eo).save('eo.png')
99
+ ```
100
+
101
+ Through the repository's own `test.py` (`--model stego_gan --no_dropout
102
+ --resnet_layer 8 --fusionblock`), outputs are written one directory per visual;
103
+ **`fake_B_clean/` is the row on this page.**
104
+
105
+ ## ⚠ `fake_B` is an oracle. `fake_B_clean` is the prediction.
106
+
107
+ The model computes two outputs:
108
+
109
+ ```
110
+ fake_B_clean = netG_A(real_A) # SAR only
111
+ fake_B = netG_A(real_A, latent_real_B.detach()) # + a feature map of the GROUND TRUTH
112
+ ```
113
+
114
+ `latent_real_B` comes from `netG_B(real_B)` — the *target* EO image. `fake_B` is
115
+ therefore conditioned on the answer and **cannot be produced at deployment**.
116
+ Only `fake_B_clean` is a prediction, and every number on this page is
117
+ `fake_B_clean`.
118
+
119
+ This is not hypothetical. Earlier in our own benchmarking the oracle was scored as
120
+ "StegoGAN" for two days on a different dataset and topped every column at
121
+ 23.19 dB, against a true deployable value of 12.74 dB. **Any published StegoGAN
122
+ number must state which of the two it is.**
123
+
124
+ ## ⚠ This cell fails the input-copy audit
125
+
126
+ We audit every cell for **identity collapse**: a model that has learned to pass
127
+ its input through rather than translate it. The test compares the mean absolute
128
+ distance from the generated image to the SAR **input** against the distance to
129
+ the EO **target**; a ratio below 1.0 means the output is closer to the input it
130
+ was given than to the answer it was asked for.
131
+
132
+ | quantity | value |
133
+ |---|---|
134
+ | mean \|generated − ground truth\| | 46.908 |
135
+ | mean \|generated − SAR input\| | 43.087 |
136
+ | **ratio** (SAR distance ÷ GT distance) | **0.919** — below 1.0 |
137
+ | verdict | **COLLAPSED** |
138
+ | cross-pair separation | 0.976 (cohort median 1.160) — no oracle behaviour |
139
+
140
+ **Read the metrics on this page with that in mind.** They are reported unchanged
141
+ because they are what the released implementation does at its own published
142
+ protocol, and a benchmark that quietly substitutes a better-behaved variant is
143
+ not reporting the method. But they should not be read as translation quality.
144
+ The separation figure is the companion check for the *opposite* failure: it is
145
+ the mean distance from a generated image to **other** images' ground truth
146
+ divided by the distance to its **own**, so a value well above the cohort median
147
+ would indicate that ground-truth information had leaked into the generation.
148
+ This cell sits at or below the median, so there is no leak — the problem here is
149
+ input-copying, and only input-copying.
150
+
151
+ ## Read before using this checkpoint
152
+
153
+ * **`--resnet_layer 8` and `--fusionblock` must match training.** They change the
154
+ architecture; omit them and the load fails.
155
+ * **`test.py` rejects `--display_id`.** That flag is declared only in the train
156
+ options. Passing it to `test.py` aborts with an unrecognised argument — and in
157
+ a shell `&&` chain the cell then reports success with an empty output
158
+ directory. This silently destroyed several of our own test passes once.
159
+ * Build with `gpu_ids=[]`; the checkpoints have bare keys.
160
+ * Normalisation is `[-1, 1]`.
161
+ * StegoGAN is **unpaired**.
162
+
163
+ ## Licence — stated factually ⚠ no upstream licence exists
164
+
165
+ **The upstream code base publishes no licence.**
166
+ [`sian-wusidi/StegoGAN`](https://github.com/sian-wusidi/StegoGAN) — the official
167
+ CVPR 2024 repository — has no LICENSE, LICENCE, COPYING or NOTICE file at any
168
+ depth; **0 of its 52 tracked files** at the pinned commit are licence-shaped; its
169
+ README has no licence section; and the GitHub API reports no declared licence,
170
+ with the `/license` endpoint returning 404. Checked 2026-08-28.
171
+
172
+ Under default copyright that means **all rights are reserved by the authors and
173
+ no express permission to redistribute derived work has been granted** to us or
174
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
175
+ end to end, and we state the position plainly rather than implying a permission
176
+ that does not exist. If you intend to redistribute this checkpoint or build on
177
+ it, assess that for yourself, and consider asking the authors directly.
178
+
179
+ **What is known about the lineage.** StegoGAN's README thanks
180
+ `junyanz/pytorch-CycleGAN-and-pix2pix`, and the parts of the tree that are
181
+ unmodified CycleGAN carry that project's BSD notice, reproduced here as
182
+ `licenses/LICENSE-cyclegan.txt`. **That notice does not extend to StegoGAN's own
183
+ contributions** — the mismatch-mask generators, the fusion block and the
184
+ consistency and regularisation losses, which are exactly what makes
185
+ `net_G_A.pth` what it is.
186
+
187
+ Please cite: Wu, Chen, Mermet, Hurni, Schindler, Gonthier and Landrieu,
188
+ *StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image
189
+ Translation*, CVPR 2024.
190
+ The full record of what we checked, per method, is in
191
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
192
+
193
+ ---
194
+
195
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
196
+ checkpoint that we retrained ourselves on QXS-SAROPT; it is not ReFlowSET
197
+ itself. Every comparison method in this repository was retrained by us on the
198
+ same splits at the same resolution and scored through one evaluation pipeline,
199
+ so the rows are directly comparable to each other — and, for the same reason,
200
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/bbdm/README.md ADDED
@@ -0,0 +1,138 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BBDM — SAR2Opt
2
+
3
+ **BBDM: Image-to-Image Translation with Brownian Bridge Diffusion Models**, Li,
4
+ Xue, Liu and Lai, *CVPR 2023*
5
+ ([arXiv:2205.07680](https://arxiv.org/abs/2205.07680)). Upstream code:
6
+ [`xuekt98/BBDM`](https://github.com/xuekt98/BBDM) at commit
7
+ `02c3b13c9f9dfab0853e32123100680a0640c4ed`.
8
+
9
+ This is the latent Brownian-bridge diffusion model we retrained ourselves on
10
+ SAR2Opt, and it is the checkpoint behind the BBDM row of the ReFlowSET
11
+ comparison table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `last_model.pth` | 2,118,317,140 | the whole trained system — see below |
18
+
19
+ `last_model.pth` is a pickled dictionary, not a bare state dict. It holds:
20
+
21
+ * `['model']` — the full `LatentBrownianBridgeModel` state dict: the denoising
22
+ UNet **and**, embedded inside it, the frozen vq-f4 VQGAN submodule;
23
+ * `['ema']` — the EMA shadow weights. **Sampling uses these**, not `['model']`;
24
+ the runner calls `apply_ema()` before generating;
25
+ * `['step']` and `['epoch']`.
26
+
27
+ UNet: latent resolution 128×128, `model_channels` 128, `num_res_blocks` 2,
28
+ attention at resolutions 32/16/8, `channel_mult` (1, 4, 8), 8 heads,
29
+ 64 channels per head, scale-shift normalisation, resblock up/downsampling.
30
+
31
+ The denoising UNet has `condition_key: nocond` — there is no separate
32
+ conditioning branch, because in BBDM the SAR image *is* one endpoint of the
33
+ Brownian bridge and the EO image is the other. The translation is the bridge,
34
+ not a conditioner.
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | **generator updates released** | **50,137** |
41
+ | batch size / resolution | 8 @ 512 px |
42
+ | iterations per epoch | 181 |
43
+ | epochs | 280 configured, capped by the step limit |
44
+ | optimizer | Adam, lr 1e-4, β₁ 0.9, no weight decay; `ReduceLROnPlateau` (factor 0.5, patience 3,000, min lr 5e-7) |
45
+ | EMA | decay 0.995, started at step 30,000, updated every 8 steps |
46
+ | bridge | `mt_type` linear, objective `grad`, L1 loss, T = 1,000, `max_var` 1.0 |
47
+ | sampler (test) | 200 skip-sampled steps, η = 1.0, `clip_denoised` off |
48
+ | latent space | CompVis **vq-f4** VQGAN, frozen (`embed_dim` 3, `n_embed` 8192) |
49
+ | horizontal flip | enabled (horizontal) |
50
+
51
+ The 50,137 rather than a round 50,000 is not a typo: the runner stops at the first epoch boundary past the step cap.
52
+
53
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
54
+
55
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
56
+ |---|---|---|---|---|
57
+ | 143.1 | 0.290 | 0.590 | 0.276 | 15.29 |
58
+
59
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
60
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
61
+ method's training and in the evaluation. No subsampling: every metric on this
62
+ page is measured over all 627 pairs.
63
+
64
+ These are our own re-evaluation numbers, measured by us on the images this
65
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
66
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
67
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
68
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
69
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
70
+ convention gives a systematically different number and the two must never be
71
+ mixed, or compared against a paper that used the other one.
72
+
73
+ ## Load it and translate one SAR image
74
+
75
+ There is no smaller entry point than the repository's own runner, because the
76
+ model is constructed with a VQGAN whose weights are loaded from a path in the
77
+ config **even though the same weights are already inside `last_model.pth`**.
78
+ You must have the CompVis latent-diffusion **vq-f4** checkpoint
79
+ (`model.ckpt`, 756,175,527 bytes) on disk and pointed at by
80
+ `model.VQGAN.params.ckpt_path`, or construction fails before any of this runs.
81
+
82
+ ```bash
83
+ # from the BBDM repository root, with a config whose data.dataset_config
84
+ # points at your SAR / EO directories and whose VQGAN ckpt_path exists:
85
+ python main.py -c configs/<your>-LBBDM-f4.yaml --gpu_ids 0 \
86
+ -r <output dir> --sample_to_eval --resume_model last_model.pth
87
+ ```
88
+
89
+ `--resume_model` sets `model.model_load_path`; the runner loads it with
90
+ `torch.load(..., weights_only=False)`, restores the EMA shadow and applies it
91
+ before sampling.
92
+
93
+ Outputs land in
94
+ `<output dir>/<dataset_name>/LBBDM-f4/sample_to_eval/`,
95
+ in three sibling directories: **`200/` holds the generated EO** (the name is the
96
+ `sample_step` count), `condition/` holds the SAR inputs and `ground_truth/` holds
97
+ the targets. Score `200/`.
98
+
99
+ ## Read before using this checkpoint
100
+
101
+ * **Sampling must use the EMA weights.** `['model']` and `['ema']` are both in
102
+ the file and they are different networks. The reported numbers are the EMA.
103
+ * **You need the external vq-f4 VQGAN.** It is not redistributed here — it is
104
+ CompVis latent-diffusion material with its own (MIT) licence, and the config
105
+ requires it at construction time regardless.
106
+ * **`torch.load` needs `weights_only=False`.** PyTorch �� 2.6 flipped that
107
+ default; these are pickled Lightning-style checkpoints. We made that change in
108
+ four places in the repository (`model/VQGAN/vqgan.py`, `runners/BaseRunner.py`
109
+ twice, `runners/.../BBDMRunner.py`) and also removed a `verbose=True` kwarg
110
+ from `ReduceLROnPlateau` that PyTorch 2.x no longer accepts. All four are
111
+ compatibility fixes; none changes the maths.
112
+ * Passes the identity-collapse audit on both datasets.
113
+
114
+ ## Licence — stated factually
115
+
116
+ Upstream `xuekt98/BBDM` ships a LICENSE file, reproduced verbatim here as
117
+ `licenses/LICENSE-bbdm.txt`: the **MIT licence**, `Copyright (c) 2023 xuekt98`.
118
+
119
+ **Redistribution of these weights is permitted.** No non-commercial clause, no
120
+ share-alike clause.
121
+
122
+ One dependency note that is attribution, not restriction: this checkpoint cannot
123
+ be run without the CompVis latent-diffusion **vq-f4** VQGAN, which is MIT and is
124
+ **not** redistributed here.
125
+
126
+ Attribution: *trained with `xuekt98/BBDM` (MIT, Copyright (c) 2023 xuekt98).
127
+ Requires the CompVis latent-diffusion vq-f4 VQGAN (MIT,
128
+ https://github.com/CompVis/latent-diffusion), which is not redistributed here.
129
+ Full text in `licenses/LICENSE-bbdm.txt`.*
130
+
131
+ ---
132
+
133
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
134
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
135
+ itself. Every comparison method in this repository was retrained by us on the
136
+ same splits at the same resolution and scored through one evaluation pipeline,
137
+ so the rows are directly comparable to each other — and, for the same reason,
138
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/cbbdm/README.md ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # cBBDM — SAR2Opt
2
+
3
+ **Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image
4
+ Translation**, Kim and Chung, *IEEE Geoscience and Remote Sensing Letters*, 2025
5
+ ([arXiv:2408.07947](https://arxiv.org/abs/2408.07947); the venue and year are
6
+ taken from the authors' own citation block — the repository records no DOI).
7
+ Upstream code:
8
+ [`egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical`](https://github.com/egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical)
9
+ at commit `8ce15934f4d4e3f01efe70d11e2d9b9e0859210c`, a fork of
10
+ [`xuekt98/BBDM`](https://github.com/xuekt98/BBDM).
11
+
12
+ This is the conditional Brownian-bridge latent diffusion model we retrained
13
+ ourselves on SAR2Opt, and it is the checkpoint behind the cBBDM row of the
14
+ ReFlowSET comparison table.
15
+
16
+ ## What is in this folder
17
+
18
+ | file | bytes | what it is |
19
+ |---|---|---|
20
+ | `last_model.pth` | 2,118,345,622 | the whole trained system — see below |
21
+
22
+ `last_model.pth` is a pickled dictionary, not a bare state dict. It holds:
23
+
24
+ * `['model']` — the full `LatentBrownianBridgeModel` state dict: the denoising
25
+ UNet **and**, embedded inside it, the frozen vq-f4 VQGAN submodule;
26
+ * `['ema']` — the EMA shadow weights. **Sampling uses these**, not `['model']`;
27
+ the runner calls `apply_ema()` before generating;
28
+ * `['step']` and `['epoch']`.
29
+
30
+ UNet: latent resolution 128×128, `model_channels` 128, `num_res_blocks` 2,
31
+ attention at resolutions 32/16/8, `channel_mult` (1, 4, 8), 8 heads,
32
+ 64 channels per head, scale-shift normalisation, resblock up/downsampling.
33
+
34
+ The denoising UNet differs from plain BBDM in exactly two ways, and they are
35
+ the method: **`in_channels` is 6, not 3**, and **`condition_key` is
36
+ `SpatialRescaler`** — a small trained module that rescales the SAR latent and
37
+ concatenates it to the bridge state at every step, so the SAR conditioning is
38
+ explicit rather than implicit in the bridge endpoints. The SpatialRescaler is
39
+ trained (it is in the model's parameter list) and is inside this file.
40
+
41
+ ## Training budget we used
42
+
43
+ | | |
44
+ |---|---|
45
+ | **generator updates released** | **50,137** |
46
+ | batch size / resolution | 8 @ 512 px |
47
+ | iterations per epoch | 181 |
48
+ | epochs | 280 configured, capped by the step limit |
49
+ | optimizer | Adam, lr 1e-4, β₁ 0.9, no weight decay; `ReduceLROnPlateau` (factor 0.5, patience 3,000, min lr 5e-7) |
50
+ | EMA | decay 0.995, started at step 30,000, updated every 8 steps |
51
+ | bridge | `mt_type` linear, objective `grad`, L1 loss, T = 1,000, `max_var` 1.0 |
52
+ | sampler (test) | 200 skip-sampled steps, η = 1.0, `clip_denoised` off |
53
+ | latent space | CompVis **vq-f4** VQGAN, frozen (`embed_dim` 3, `n_embed` 8192) |
54
+ | horizontal flip | enabled (horizontal) |
55
+
56
+ The 50,137 rather than a round 50,000 is not a typo: the runner stops at the first epoch boundary past the step cap.
57
+
58
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
59
+
60
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
61
+ |---|---|---|---|---|
62
+ | 222.3 | 0.377 | 0.571 | 0.361 | 17.05 |
63
+
64
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
65
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
66
+ method's training and in the evaluation. No subsampling: every metric on this
67
+ page is measured over all 627 pairs.
68
+
69
+ These are our own re-evaluation numbers, measured by us on the images this
70
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
71
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
72
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
73
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
74
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
75
+ convention gives a systematically different number and the two must never be
76
+ mixed, or compared against a paper that used the other one.
77
+
78
+ ## Load it and translate one SAR image
79
+
80
+ There is no smaller entry point than the repository's own runner, because the
81
+ model is constructed with a VQGAN whose weights are loaded from a path in the
82
+ config **even though the same weights are already inside `last_model.pth`**.
83
+ You must have the CompVis latent-diffusion **vq-f4** checkpoint
84
+ (`model.ckpt`, 756,175,527 bytes) on disk and pointed at by
85
+ `model.VQGAN.params.ckpt_path`, or construction fails before any of this runs.
86
+
87
+ ```bash
88
+ # from the cBBDM repository root, with a config whose data.dataset_config
89
+ # points at your SAR / EO directories and whose VQGAN ckpt_path exists:
90
+ python main.py -c configs/<your>-cBBDM-f4.yaml --gpu_ids 0 \
91
+ -r <output dir> --sample_to_eval --resume_model last_model.pth
92
+ ```
93
+
94
+ `--resume_model` sets `model.model_load_path`; the runner loads it with
95
+ `torch.load(..., weights_only=False)`, restores the EMA shadow and applies it
96
+ before sampling.
97
+
98
+ Outputs land in
99
+ `<output dir>/<dataset_name>/cBBDM-f4/sample_to_eval/`,
100
+ in three sibling directories: **`200/` holds the generated EO** (the name is the
101
+ `sample_step` count), `condition/` holds the SAR inputs and `ground_truth/` holds
102
+ the targets. Score `200/`.
103
+
104
+ ## Read before using this checkpoint
105
+
106
+ * **Sampling must use the EMA weights.** `['model']` and `['ema']` are both in
107
+ the file and they are different networks. The reported numbers are the EMA.
108
+ * **You need the external vq-f4 VQGAN.** It is not redistributed here — it is
109
+ CompVis latent-diffusion material with its own (MIT) licence, and the config
110
+ requires it at construction time regardless.
111
+ * **`torch.load` needs `weights_only=False`.** PyTorch ≥ 2.6 flipped that
112
+ default; these are pickled Lightning-style checkpoints. We made that change in
113
+ four places in the repository (`model/VQGAN/vqgan.py`, `runners/BaseRunner.py`
114
+ twice, `runners/.../BBDMRunner.py`) and also removed a `verbose=True` kwarg
115
+ from `ReduceLROnPlateau` that PyTorch 2.x no longer accepts. All four are
116
+ compatibility fixes; none changes the maths.
117
+ * **A performance patch we applied, and when.** This fork regressed the
118
+ DataLoader worker count from upstream BBDM's 8 to 0, which dominated wall-clock
119
+ time (one 32-pair batch: 6.5 s serial versus 1.1 s across 8 threads). We
120
+ restored `num_workers=8`. **Both cells published here were trained *before*
121
+ that change, i.e. with 0 workers** — same data, same order, same step count,
122
+ only slower. The numbers on this page are unaffected either way: worker count
123
+ cannot change which samples the model sees.
124
+ * Passes the identity-collapse audit on both datasets.
125
+
126
+ ## Licence — stated factually
127
+
128
+ Upstream `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical` ships a LICENSE file,
129
+ reproduced verbatim here as `licenses/LICENSE-cbbdm.txt`: the **MIT licence**,
130
+ `Copyright (c) 2025 egshkim`. Its README adds: *"The original source code is
131
+ licensed under the MIT License, and this repository continues to comply with its
132
+ terms"* — the original being `xuekt98/BBDM`, also MIT
133
+ (`licenses/LICENSE-bbdm.txt`).
134
+
135
+ **Redistribution of these weights is permitted.** No non-commercial clause, no
136
+ share-alike clause.
137
+
138
+ One dependency note that is attribution, not restriction: this checkpoint cannot
139
+ be run without the CompVis latent-diffusion **vq-f4** VQGAN, which is MIT and is
140
+ **not** redistributed here.
141
+
142
+ Attribution: *trained with `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical`
143
+ (MIT, Copyright (c) 2025 egshkim), a fork of `xuekt98/BBDM` (MIT, Copyright (c)
144
+ 2023 xuekt98). Requires the CompVis vq-f4 VQGAN (MIT,
145
+ https://github.com/CompVis/latent-diffusion), not redistributed here. Full texts
146
+ in `licenses/LICENSE-cbbdm.txt` and `licenses/LICENSE-bbdm.txt`.*
147
+
148
+ ---
149
+
150
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
151
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
152
+ itself. Every comparison method in this repository was retrained by us on the
153
+ same splits at the same resolution and scored through one evaluation pipeline,
154
+ so the rows are directly comparable to each other — and, for the same reason,
155
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/cdiffset/README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # C-DiffSET — SAR2Opt
2
+
3
+ **C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with
4
+ Confidence-Guided Reliable Object Generation**, Do, Lee, Lee and Kim,
5
+ *IEEE Transactions on Circuits and Systems for Video Technology*, 2026
6
+ ([doi:10.1109/TCSVT.2026.3701447](https://doi.org/10.1109/TCSVT.2026.3701447),
7
+ [arXiv:2411.10788](https://arxiv.org/abs/2411.10788)). Upstream code:
8
+ [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET).
9
+
10
+ This is the **stage-2 (confidence-guided) UNet** we retrained ourselves on
11
+ SAR2Opt, and it is the checkpoint behind the C-DiffSET row of the ReFlowSET
12
+ comparison table.
13
+
14
+ ## What is in this folder
15
+
16
+ A single `UNet2DConditionModel` (865,925,125 parameters, fp32, 686 tensors) with
17
+ two channel modifications relative to stock SD 2.1-base:
18
+
19
+ * **`conv_in` 4 -> 8 channels.** The UNet is conditioned by channel
20
+ concatenation in latent space, **SAR first**:
21
+ `torch.cat([sar_latent, noisy_eo_latent], dim=1)`. The extra half was
22
+ initialised by duplicating the pretrained kernel and halving it; in this
23
+ trained checkpoint the two halves have clearly diverged
24
+ (mean |w| 0.01517 on channels 0-3 vs 0.02702 on channels 4-7).
25
+ * **`conv_out` 4 -> 5 channels.** Channels 0-3 are the usual epsilon
26
+ prediction; channel 4 is a **raw spatial variance** used by the
27
+ confidence-guided loss. It was zero-initialised and is no longer zero
28
+ (mean |w| 0.03187), i.e. the head really trained. To turn it into a variance:
29
+ `var = clamp(softplus(raw + c) + 1e-6, 1e-6, 10)` where the offset
30
+ `c = log(exp(1 - 1e-6) - 1)` is the inverse softplus of the initial variance,
31
+ and confidence is `1 / var`. **The scheduler must only ever see channels 0-3.**
32
+
33
+ There is no VAE, no text encoder and no scheduler here — take those from the
34
+ base model (below).
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | optimizer updates released | **40,000** (fixed-step snapshot `checkpoint-40000`) |
41
+ | updates the run was configured for | 50,000 |
42
+ | initialised from | our own stage-1 checkpoint (the `sd21ft` folder's run, at its final 50,000-update state) |
43
+ | batch size / resolution | 16 @ 512 px |
44
+ | epochs the 40,000 updates work out to | ~444 (1,450 training pairs, 90 updates/epoch) |
45
+ | optimizer | AdamW, lr 3e-5, weight decay 0.01, cosine schedule, 100-step warmup |
46
+ | precision | fp32 (no mixed precision) |
47
+ | seed | 2024 |
48
+ | prediction type | epsilon, with the confidence-guided (Gaussian-NLL) loss |
49
+ | prompt | the fixed string `"electro-optical image"` |
50
+ | augmentation | horizontal flip, vertical flip, 90-degree rotation |
51
+ | test-time sampler | **DDIM, 50 steps**, fp32, one image at a time |
52
+
53
+ The budget is quoted in **optimizer updates**, not epochs, because the two
54
+ datasets have very different training-set sizes. Two things a reader
55
+ reproducing this needs to know: the released snapshot is taken at update 40,000
56
+ of a cosine schedule that was laid out for 50,000, so the learning rate had
57
+ **not** finished decaying at that point; and we publish the fixed-step snapshot
58
+ rather than the repository's validation-selected `best/`, because every other
59
+ method in our comparison publishes its last checkpoint and selecting on
60
+ validation PSNR would have been an asymmetry in C-DiffSET's favour.
61
+
62
+ Data: 512 px. The source images are 600 px and are **never resized**: training takes a random 512 crop, evaluation takes the centre 512 crop (offset 44). The EO/SAR pairing follows the official SAR2Opt layout (`trainB`/`testB` are EO, `trainA`/`testA` are SAR).
63
+
64
+ ## Base model
65
+
66
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
67
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
68
+ available on the Hub, our training and every snippet below load the community
69
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
70
+ That is what we actually trained against — verified in the training
71
+ configuration files, the launch scripts and the training logs, none of which
72
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
73
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
74
+ expected to work but has not been verified by us.
75
+
76
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
77
+
78
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
79
+ |---|---|---|---|---|---|
80
+ | 78.1 | 0.0139 | 0.214 | 0.529 | 16.81 | 0.314 |
81
+
82
+ These are our own re-evaluation numbers, not numbers copied from any paper.
83
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
84
+ convention); the alternative [0, 1] convention gives a systematically different
85
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
86
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
87
+
88
+ > **DISTS on this cell was re-measured.** This cell's prediction dump was regenerated after the only extended-metric pass that scored it, and that pass resumes on the *number* of images rather than their content, so its stored DISTS went stale. A full re-measurement of the current dump at n=627 gives the value printed above; the harness was validated on a cell that is not stale, whose DISTS it reproduces to 6e-6. PSNR, SSIM, FID and LPIPS come from the headline pass, which rescores every cell on every run.
89
+
90
+ ## Load it and translate one SAR PNG
91
+
92
+ ```python
93
+ import numpy as np
94
+ import torch
95
+ from PIL import Image
96
+ from diffusers import UNet2DConditionModel, AutoencoderKL, DDIMScheduler
97
+ from transformers import CLIPTextModel, CLIPTokenizer
98
+
99
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
100
+ BASE = "Manojb/stable-diffusion-2-1-base" # frozen VAE / text encoder / scheduler
101
+ DEV, DT = "cuda", torch.float32
102
+
103
+ unet = UNet2DConditionModel.from_pretrained(
104
+ REPO, subfolder="baselines/sar2opt/cdiffset", torch_dtype=DT).to(DEV).eval()
105
+ vae = AutoencoderKL.from_pretrained(BASE, subfolder="vae", torch_dtype=DT).to(DEV).eval()
106
+ tok = CLIPTokenizer.from_pretrained(BASE, subfolder="tokenizer")
107
+ txt = CLIPTextModel.from_pretrained(BASE, subfolder="text_encoder", torch_dtype=DT).to(DEV).eval()
108
+ sch = DDIMScheduler.from_pretrained(BASE, subfolder="scheduler")
109
+ sch.set_timesteps(50, device=DEV)
110
+
111
+ # The fixed training prompt, tokenized exactly as in training (no padding).
112
+ ids = tok("electro-optical image", padding="do_not_pad",
113
+ max_length=tok.model_max_length, truncation=True,
114
+ return_tensors="pt").input_ids.to(DEV)
115
+ with torch.no_grad():
116
+ embed = txt(ids)[0]
117
+
118
+ # SAR input: 512x512 PNG -> RGB -> [-1, 1]. Do not resize; crop if needed.
119
+ sar = Image.open("sar.png").convert("RGB")
120
+ x = torch.from_numpy(np.array(sar)).permute(2, 0, 1)[None]
121
+ x = (x.to(DEV, DT) / 255.0) * 2 - 1
122
+
123
+ with torch.no_grad():
124
+ sar_lat = vae.encode(x).latent_dist.mean * vae.config.scaling_factor # .mean, not .sample()
125
+ eo_lat = torch.randn_like(sar_lat)
126
+ for t in sch.timesteps:
127
+ out = unet(torch.cat([sar_lat, eo_lat], 1), # SAR first, then the noisy EO latent
128
+ t, encoder_hidden_states=embed).sample
129
+ # channels 0-3 are epsilon; channel 4 is the raw variance and never
130
+ # goes to the scheduler.
131
+ eo_lat = sch.step(out[:, :4], t, eo_lat).prev_sample
132
+ eo = vae.decode(eo_lat / vae.config.scaling_factor).sample
133
+ eo = (eo * 0.5 + 0.5).clamp(0, 1)[0].cpu()
134
+
135
+ Image.fromarray((eo.permute(1, 2, 0).numpy() * 255).round().astype(np.uint8)).save("eo.png")
136
+ ```
137
+
138
+ The three things that silently produce a plausible-but-wrong image if you get
139
+ them backwards: the concatenation order is **SAR first**, only `out[:, :4]`
140
+ reaches the scheduler, and the VAE must be read as `latent_dist.mean`, not
141
+ `.sample()`.
142
+
143
+ ## Licence status — stated factually
144
+
145
+ * **Training code:** MIT, `Copyright (c) 2026 KAIST VICLab`
146
+ ([`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET) ships a
147
+ LICENSE file).
148
+ * **These weights:** they are a fine-tune of Stable Diffusion 2.1-base and are
149
+ therefore a *Derivative of the Model* under the **CreativeML Open RAIL++-M**
150
+ licence. The base repository we loaded declares `openrail++`.
151
+ The **Attachment A use restrictions travel with these weights** — they bind
152
+ you, and you must pass them on to anyone you redistribute to. The full text,
153
+ including Attachment A, ships in this repository as `LICENSE-WEIGHTS.md`.
154
+
155
+ Attribution: *fine-tuned from Stable Diffusion 2.1-base (obtained as
156
+ `Manojb/stable-diffusion-2-1-base`, a re-upload of
157
+ `stabilityai/stable-diffusion-2-1-base`) with the training code of
158
+ `KAIST-VICLab/C-DiffSET`. Do, Lee, Lee and Kim, "C-DiffSET", IEEE TCSVT 2026,
159
+ doi:10.1109/TCSVT.2026.3701447.*
160
+
161
+ ---
162
+
163
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
164
+ checkpoint retrained by us on SAR2Opt; it is not ReFlowSET itself. Every
165
+ comparison method in the release was retrained by us on the same splits at the
166
+ same resolution, so the rows are directly comparable to each other.
baselines/sar2opt/cdiffset/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "UNet2DConditionModel",
3
+ "_diffusers_version": "0.10.0.dev0",
4
+ "act_fn": "silu",
5
+ "attention_head_dim": [
6
+ 5,
7
+ 10,
8
+ 20,
9
+ 20
10
+ ],
11
+ "block_out_channels": [
12
+ 320,
13
+ 640,
14
+ 1280,
15
+ 1280
16
+ ],
17
+ "center_input_sample": false,
18
+ "cross_attention_dim": 1024,
19
+ "down_block_types": [
20
+ "CrossAttnDownBlock2D",
21
+ "CrossAttnDownBlock2D",
22
+ "CrossAttnDownBlock2D",
23
+ "DownBlock2D"
24
+ ],
25
+ "downsample_padding": 1,
26
+ "dual_cross_attention": false,
27
+ "flip_sin_to_cos": true,
28
+ "freq_shift": 0,
29
+ "in_channels": 8,
30
+ "layers_per_block": 2,
31
+ "mid_block_scale_factor": 1,
32
+ "norm_eps": 1e-05,
33
+ "norm_num_groups": 32,
34
+ "num_class_embeds": null,
35
+ "only_cross_attention": false,
36
+ "out_channels": 5,
37
+ "sample_size": 64,
38
+ "up_block_types": [
39
+ "UpBlock2D",
40
+ "CrossAttnUpBlock2D",
41
+ "CrossAttnUpBlock2D",
42
+ "CrossAttnUpBlock2D"
43
+ ],
44
+ "use_linear_projection": true
45
+ }
baselines/sar2opt/conddiff/README.md ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Conditional Diffusion — SAR2Opt
2
+
3
+ **Conditional Diffusion for SAR to Optical Image Translation**, Bai, Pu and Xu,
4
+ *IEEE Geoscience and Remote Sensing Letters*, 2023
5
+ ([doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143); the
6
+ year is the one in the authors' own citation block — a table that labels this
7
+ row with a later issue year is referring to the same paper).
8
+
9
+ Upstream code:
10
+ [`Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation`](https://github.com/Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation),
11
+ a modified copy of [`openai/guided-diffusion`](https://github.com/openai/guided-diffusion).
12
+ **The exact upstream commit is not recoverable** — see the licence section.
13
+
14
+ This is the pixel-space conditional diffusion model we retrained ourselves on
15
+ SAR2Opt, and it is the checkpoint behind the Conditional Diffusion row of the
16
+ ReFlowSET comparison table.
17
+
18
+ ## What is in this folder
19
+
20
+ | file | bytes | what it is |
21
+ |---|---|---|
22
+ | `ema_final.pt` | 662,459,343 | EMA (decay 0.9999) of the UNet at update 50,000 — a **bare state dict**, no wrapper key |
23
+
24
+ `ema_final.pt` is the EMA of a **164.3 M-parameter guided-diffusion UNet**:
25
+ `num_channels` 128, `num_res_blocks` 3, `learn_sigma` **False**, attention at
26
+ resolutions 16 and 8. It is a plain `state_dict` — load it directly, with no
27
+ `['model']` or `['ema']` indirection.
28
+
29
+ The QXS-SAROPT and SAR2Opt files differ in size (657,495,287 vs 662,459,343
30
+ bytes) purely because of the 256 px versus 512 px positional and attention
31
+ buffers; the parameter count is otherwise the same.
32
+
33
+ **Conditioning.** The SAR image is concatenated to the noisy state **noise-free**
34
+ at every reverse step and at training time — that is the paper's claim, and the
35
+ code does exactly that.
36
+
37
+ ## Training budget we used
38
+
39
+ | | |
40
+ |---|---|
41
+ | **generator updates released** | **50,000** |
42
+ | batch size / resolution | 6 @ 512 px |
43
+ | optimizer | Adam, **constant** lr 1e-4 with `--lr_anneal_steps 50000` (linear decay to zero, which is also the only stop mechanism in the released code) |
44
+ | EMA | decay 0.9999 |
45
+ | diffusion | T = 2,000, linear β schedule, eps-prediction, `learn_sigma` False |
46
+ | sampler (test) | **respaced DDPM, 250 steps**, `clip_denoised` on |
47
+ | augmentation | none — the released code has none |
48
+ | input | deterministic centre 512 crops of the 600 px tiles |
49
+
50
+ **The paper and the released code disagree about the learning-rate schedule.**
51
+ The paper describes warmup plus cosine; the released code implements neither. We
52
+ ran the code.
53
+
54
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
55
+
56
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
57
+ |---|---|---|---|---|
58
+ | 211.8 | 0.415 | 0.686 | 0.248 | 12.48 |
59
+
60
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
61
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
62
+ method's training and in the evaluation. No subsampling: every metric on this
63
+ page is measured over all 627 pairs.
64
+
65
+ These are our own re-evaluation numbers, measured by us on the images this
66
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
67
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
68
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
69
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
70
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
71
+ convention gives a systematically different number and the two must never be
72
+ mixed, or compared against a paper that used the other one.
73
+
74
+ ## Load it and translate one SAR image
75
+
76
+ ```python
77
+ import torch
78
+ from guided_diffusion.script_util import (create_model_and_diffusion,
79
+ model_and_diffusion_defaults)
80
+
81
+ d = model_and_diffusion_defaults()
82
+ d.update(image_size=512, num_channels=128, num_res_blocks=3, learn_sigma=False,
83
+ diffusion_steps=2000, noise_schedule='linear', timestep_respacing='250')
84
+ model, diffusion = create_model_and_diffusion(**d)
85
+ model.load_state_dict(torch.load('ema_final.pt', map_location='cpu'))
86
+ model.cuda().eval()
87
+
88
+ # sar: (1, 3, 512, 512) float tensor in [-1, 1]
89
+ sample = diffusion.p_sample_loop(model, (1, 3, 512, 512),
90
+ clip_denoised=True, model_kwargs={},
91
+ noise=None, condition=sar)
92
+ ```
93
+
94
+ The repository imports `blobfile` and `mpi4py` unconditionally; a single-process
95
+ run on a local filesystem needs either those packages or small local stand-ins on
96
+ `PYTHONPATH`.
97
+
98
+ **Never pass `use_ddim=True`.** See below.
99
+
100
+ ## Read before using this checkpoint
101
+
102
+ * **DDIM is broken upstream, and that is not a choice we made.**
103
+ `p_sample_loop(..., condition=None, ...)` accepts and threads the SAR
104
+ condition; `ddim_sample_loop(...)` has **no `condition` parameter at all**, so
105
+ passing `--use_ddim True` raises `TypeError`. Sampling is respaced DDPM with
106
+ 250 steps, which is also what the authors' own `sample.sh` uses.
107
+ * **A correctness bug in the released sampler, which we fixed.** Upstream paired
108
+ each SAR image with an EO image by **unsorted `os.listdir` position** — i.e. by
109
+ filesystem order. Any number produced with the unpatched sampler is measured
110
+ against effectively arbitrary ground truth. We sort both listings. If you
111
+ reproduce this row from a clean upstream checkout, apply that fix or your
112
+ metrics are meaningless.
113
+ * **Three further changes we made**, all commented in place: the
114
+ distributed-init helper no longer overwrites `CUDA_VISIBLE_DEVICES` (upstream
115
+ pins rank % 8, which on a shared machine hijacks another user's device); the
116
+ noise tensor for a partial last batch is shaped from the batch rather than from
117
+ the `--batch_size` flag; and the sampler takes explicit input/output directories
118
+ and builds its resize transform at run time, because the module-level transform
119
+ hard-codes 256 px and would silently downsize the 512 px cell.
120
+ * **The training loader requires integer filenames.** It sorts with
121
+ `int(stem)`, so any non-numeric stem raises `ValueError`. Feed it an
122
+ integer-named adapter directory and keep a manifest to map back to the real
123
+ stems.
124
+ * **Do not let the released 512 px path resize.** The fork's `center_crop_arr`
125
+ *resizes* 600 → 512. We wrote deterministic centre-512 crops instead, to keep a
126
+ crop-not-resize protocol across the whole benchmark.
127
+ * Passes the identity-collapse audit on both datasets.
128
+
129
+ ## Licence — stated factually ⚠ no upstream licence exists
130
+
131
+ **The upstream code base publishes no licence.**
132
+ [`Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation`](https://github.com/Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation)
133
+ has no LICENSE, LICENCE, COPYING or NOTICE file anywhere in the tree we trained
134
+ from, no licence section in its README, and the GitHub API reports no declared
135
+ licence, with the `/license` endpoint returning 404. Checked 2026-08-28.
136
+
137
+ Under default copyright that means **all rights are reserved by the authors and
138
+ no express permission to redistribute derived work has been granted** to us or
139
+ to you. We publish this checkpoint anyway and state the position plainly. Assess
140
+ redistribution for yourself; consider asking the authors.
141
+
142
+ **A second gap, which is about reproducibility rather than licensing.** Our
143
+ vendored copy of this repository carries no version control and records no
144
+ upstream URL inside its tree, so **the exact commit these weights were trained
145
+ from cannot be recovered**. A "clone upstream, then apply our patch" recipe is
146
+ therefore not possible for this row.
147
+
148
+ **Lineage.** The README states the repository is based on
149
+ `openai/guided-diffusion` with modifications, and the tree is visibly that code
150
+ base — OpenAI provenance comments survive in `guided_diffusion/logger.py` and
151
+ `guided_diffusion/unet.py`. `openai/guided-diffusion` is **MIT**; the unmodified
152
+ guided-diffusion parts carry that licence, which does **not** extend to the
153
+ authors' modifications. We do not ship the guided-diffusion licence text here
154
+ because it is not vendored in the tree we trained from — take it from that
155
+ repository if you need it, and make sure the copyright line you carry is
156
+ OpenAI's rather than another project's MIT file.
157
+
158
+ **We modified the code further**, and say so as a matter of discipline rather
159
+ than because any licence compels it: the four changes listed in the section
160
+ above, of which the sorted-listing fix is a correctness fix.
161
+
162
+ Please cite: Bai, Pu and Xu, *Conditional Diffusion for SAR to Optical Image
163
+ Translation*, IEEE Geoscience and Remote Sensing Letters,
164
+ [doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143).
165
+ The full record of what we checked, per method, is in
166
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
167
+
168
+ ---
169
+
170
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
171
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
172
+ itself. Every comparison method in this repository was retrained by us on the
173
+ same splits at the same resolution and scored through one evaluation pipeline,
174
+ so the rows are directly comparable to each other — and, for the same reason,
175
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/controlnet/README.md ADDED
@@ -0,0 +1,128 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ControlNet — SAR2Opt
2
+
3
+ **Adding Conditional Control to Text-to-Image Diffusion Models**, Zhang, Rao and
4
+ Agrawala, *ICCV* 2023. The SAR image is the control signal and the base
5
+ Stable Diffusion 2.1-base UNet stays frozen.
6
+
7
+ > **How our row was produced, precisely.** We trained with the
8
+ > `examples/controlnet/train_controlnet.py` script from
9
+ > [`huggingface/diffusers`](https://github.com/huggingface/diffusers)
10
+ > (Apache-2.0, Copyright 2025 The HuggingFace Inc. team), initialising the
11
+ > adapter with `ControlNetModel.from_unet` on SD 2.1-base. **No code from
12
+ > `lllyasviel/ControlNet` was used.** The method is theirs; the implementation
13
+ > is the diffusers one.
14
+
15
+ ## What is in this folder
16
+
17
+ A complete diffusers `ControlNetModel` folder — `config.json` plus
18
+ `diffusion_pytorch_model.safetensors` (364,228,240 parameters, fp32, 340
19
+ tensors). It is the encoder half of the UNet plus the zero-convolutions and the
20
+ conditioning embedder; it has no `up_blocks` and no `conv_out`, so **it cannot
21
+ generate anything on its own** and must be paired with the base pipeline.
22
+
23
+ The conditioning input is 3-channel RGB (`conditioning_channels: 3`) with
24
+ embedder widths `[16, 32, 96, 256]`; the SAR PNG is opened as RGB, so the three
25
+ channels are identical.
26
+
27
+ ## Training budget we used
28
+
29
+ | | |
30
+ |---|---|
31
+ | optimizer updates | **50,000** |
32
+ | batch size / resolution | 8 @ 512 px |
33
+ | epochs the 50,000 updates work out to | ~276 (1,450 training pairs, 181 updates/epoch) |
34
+ | optimizer | AdamW, lr 1e-5, **constant** schedule (the script's default) |
35
+ | precision | bf16 mixed precision — the saved tensors are the fp32 master weights |
36
+ | seed | 42 |
37
+ | trainable parameters | the adapter only; the SD 2.1-base UNet, VAE and text encoder stayed frozen |
38
+ | prompt | the fixed string `"electro-optical image"` for every pair |
39
+ | augmentation | **none** — the diffusers trainer applies only `Resize(res)` + `CenterCrop(res)`, both no-ops at our input size, and no flips or rotations |
40
+ | test-time sampler | **UniPCMultistepScheduler, 50 steps, guidance scale 7.5**, bf16 |
41
+
42
+ Unlike the two UNet folders in this repository, this is **not** an SD 2.1
43
+ fine-tune: the base weights were never updated, which is why only the 364 M
44
+ adapter ships.
45
+
46
+ Reproduction note: our inference script creates **one** `torch.Generator`
47
+ seeded 42 and reuses it across the whole test set in batches of 8,
48
+ so a single image regenerated on its own will not be bit-identical to our dump
49
+ even at the same seed. The conditioning image is
50
+ the SAR PNG opened as RGB and bilinearly resized to 512x512 (a no-op at our
51
+ evaluation resolution, since the SAR inputs are already 512 px).
52
+
53
+ Data: 512 px. The 600 px source images were pre-cropped to a deterministic **centre** 512 crop (offset 44) for both training and evaluation, and are **never resized**. So, unlike the C-DiffSET rows in this repository, this row trained on a fixed crop rather than a random one. The EO/SAR pairing follows the official SAR2Opt layout (`trainB`/`testB` are EO, `trainA`/`testA` are SAR).
54
+
55
+ ## Base model
56
+
57
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
58
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
59
+ available on the Hub, our training and every snippet below load the community
60
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
61
+ That is what we actually trained against — verified in the training
62
+ configuration files, the launch scripts and the training logs, none of which
63
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
64
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
65
+ expected to work but has not been verified by us.
66
+
67
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
68
+
69
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
70
+ |---|---|---|---|---|---|
71
+ | 140.5 | 0.0480 | 0.350 | 0.643 | 11.73 | 0.217 |
72
+
73
+ These are our own re-evaluation numbers, not numbers copied from any paper.
74
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
75
+ convention); the alternative [0, 1] convention gives a systematically different
76
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
77
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
78
+ {}
79
+
80
+ ## Load it and translate one SAR PNG
81
+
82
+ ```python
83
+ import torch
84
+ from PIL import Image
85
+ from diffusers import (ControlNetModel, StableDiffusionControlNetPipeline,
86
+ UniPCMultistepScheduler)
87
+
88
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
89
+ BASE = "Manojb/stable-diffusion-2-1-base"
90
+
91
+ controlnet = ControlNetModel.from_pretrained(
92
+ REPO, subfolder="baselines/sar2opt/controlnet", torch_dtype=torch.bfloat16)
93
+ pipe = StableDiffusionControlNetPipeline.from_pretrained(
94
+ BASE, controlnet=controlnet, torch_dtype=torch.bfloat16,
95
+ safety_checker=None).to("cuda")
96
+ pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
97
+
98
+ sar = Image.open("sar.png").convert("RGB").resize((512, 512), Image.BILINEAR)
99
+ eo = pipe("electro-optical image", image=sar, height=512, width=512,
100
+ num_inference_steps=50, guidance_scale=7.5,
101
+ generator=torch.Generator("cuda").manual_seed(42)).images[0]
102
+ eo.save("eo.png")
103
+ ```
104
+
105
+ ## Licence status — stated factually
106
+
107
+ * **Training code:** Apache-2.0. The trainer is
108
+ `examples/controlnet/train_controlnet.py` from
109
+ [`huggingface/diffusers`](https://github.com/huggingface/diffusers),
110
+ `Copyright 2025 The HuggingFace Inc. team`; the Apache-2.0 grant is in the
111
+ file header. Our copy of that script is four loose files rather than a full
112
+ clone, so it carries no separate repository LICENSE file — the header is the
113
+ grant.
114
+ * **Method credit:** ControlNet (Zhang et al., ICCV 2023),
115
+ [`lllyasviel/ControlNet`](https://github.com/lllyasviel/ControlNet),
116
+ Apache-2.0. **No code from that repository was used in this row.**
117
+ * **These weights:** initialised from Stable Diffusion 2.1-base via
118
+ `ControlNetModel.from_unet`, so they are an SD 2.1 derivative under the
119
+ **CreativeML Open RAIL++-M** licence (`openrail++`). The **Attachment A use
120
+ restrictions travel with these weights**. Full text, including Attachment A,
121
+ ships in this repository as `LICENSE-WEIGHTS.md`.
122
+
123
+ ---
124
+
125
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
126
+ checkpoint retrained by us on SAR2Opt; it is not ReFlowSET itself. Every
127
+ comparison method in the release was retrained by us on the same splits at the
128
+ same resolution, so the rows are directly comparable to each other.
baselines/sar2opt/controlnet/config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "ControlNetModel",
3
+ "_diffusers_version": "0.37.1",
4
+ "act_fn": "silu",
5
+ "addition_embed_type": null,
6
+ "addition_embed_type_num_heads": 64,
7
+ "addition_time_embed_dim": null,
8
+ "attention_head_dim": [
9
+ 5,
10
+ 10,
11
+ 20,
12
+ 20
13
+ ],
14
+ "block_out_channels": [
15
+ 320,
16
+ 640,
17
+ 1280,
18
+ 1280
19
+ ],
20
+ "class_embed_type": null,
21
+ "conditioning_channels": 3,
22
+ "conditioning_embedding_out_channels": [
23
+ 16,
24
+ 32,
25
+ 96,
26
+ 256
27
+ ],
28
+ "controlnet_conditioning_channel_order": "rgb",
29
+ "cross_attention_dim": 1024,
30
+ "down_block_types": [
31
+ "CrossAttnDownBlock2D",
32
+ "CrossAttnDownBlock2D",
33
+ "CrossAttnDownBlock2D",
34
+ "DownBlock2D"
35
+ ],
36
+ "downsample_padding": 1,
37
+ "encoder_hid_dim": null,
38
+ "encoder_hid_dim_type": null,
39
+ "flip_sin_to_cos": true,
40
+ "freq_shift": 0,
41
+ "global_pool_conditions": false,
42
+ "in_channels": 4,
43
+ "layers_per_block": 2,
44
+ "mid_block_scale_factor": 1,
45
+ "mid_block_type": "UNetMidBlock2DCrossAttn",
46
+ "norm_eps": 1e-05,
47
+ "norm_num_groups": 32,
48
+ "num_attention_heads": null,
49
+ "num_class_embeds": null,
50
+ "only_cross_attention": false,
51
+ "projection_class_embeddings_input_dim": null,
52
+ "resnet_time_scale_shift": "default",
53
+ "transformer_layers_per_block": 1,
54
+ "upcast_attention": false,
55
+ "use_linear_projection": true
56
+ }
baselines/sar2opt/cyclegan/README.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CycleGAN — SAR2Opt
2
+
3
+ **Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial
4
+ Networks**, Zhu, Park, Isola and Efros, *ICCV 2017*
5
+ ([arXiv:1703.10593](https://arxiv.org/abs/1703.10593)). Upstream code:
6
+ [`junyanz/pytorch-CycleGAN-and-pix2pix`](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
7
+ at commit `2a7afba2895d52556dd5dfe07e8555ef657ced6f`.
8
+
9
+ This is the unpaired cycle-consistent GAN we retrained ourselves on SAR2Opt,
10
+ and it is the checkpoint behind the CycleGAN row of the ReFlowSET comparison
11
+ table. **It fails our input-copy audit — see the section below before you use
12
+ or quote it.**
13
+
14
+ ## What is in this folder
15
+
16
+ | file | bytes | what it is |
17
+ |---|---|---|
18
+ | `net_G_A.pth` | 45,533,133 | **SAR → EO** — this is the translation direction the table reports |
19
+ | `net_G_B.pth` | 45,533,133 | EO → SAR, the reverse generator that closes the cycle; not used to produce any reported number |
20
+
21
+ Both are **`resnet_9blocks` generators** with instance normalisation,
22
+ `ngf = 64`, 3 in / 3 out, dropout disabled. The direction assignment is not a
23
+ convention we chose: upstream's unaligned loader binds `A_paths` to the
24
+ `trainA` / `testA` directory, which holds the SAR side, and the model computes
25
+ `fake_B = netG_A(real_A)`. So **`net_G_A` is the SAR → EO generator**.
26
+
27
+ `net_G_B` is shipped because it is half of the trained system and someone
28
+ reproducing the cycle needs it; it is not needed to reproduce our numbers. The
29
+ two discriminators are not released.
30
+
31
+ ## Training budget we used
32
+
33
+ | | |
34
+ |---|---|
35
+ | **generator updates released** | **72,600** |
36
+ | batch size | 4 |
37
+ | epochs (constant LR + linear decay) | 100 + 100 |
38
+ | iterations per epoch | 363 (1,450 training images, the loader does not drop the last batch) |
39
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
40
+ | adversarial loss | LSGAN; discriminator `basic` (70×70 PatchGAN) |
41
+ | cycle losses | λ_A 10, λ_B 10 |
42
+ | identity loss | λ_identity 0.5 |
43
+ | augmentation | horizontal flip |
44
+ | input | random 512 crops of the 600 px tiles |
45
+
46
+ Budgets are quoted in **optimizer updates**, never epochs. This is the single
47
+ most important comparability rule for the unpaired methods in this benchmark:
48
+ "100 + 100 epochs" is 72,600 updates on 1,450 images and 400,000+ on
49
+ 16,001, a difference that reads as budget-matched and is not.
50
+
51
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
52
+
53
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
54
+ |---|---|---|---|---|
55
+ | 143.5 | 0.330 | 0.650 | 0.178 | 12.90 |
56
+
57
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
58
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
59
+ method's training and in the evaluation. No subsampling: every metric on this
60
+ page is measured over all 627 pairs.
61
+
62
+ These are our own re-evaluation numbers, measured by us on the images this
63
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
64
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
65
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
66
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
67
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
68
+ convention gives a systematically different number and the two must never be
69
+ mixed, or compared against a paper that used the other one.
70
+
71
+ ## Load it and translate one SAR image
72
+
73
+ Clone `junyanz/pytorch-CycleGAN-and-pix2pix` at the commit above and run this
74
+ from its root. The model code is stock — we applied no patch to it.
75
+
76
+ ```python
77
+ import numpy as np, torch
78
+ from PIL import Image
79
+ from models.networks import define_G # from the junyanz repo
80
+
81
+ G_A = define_G(3, 3, 64, 'resnet_9blocks', 'instance', use_dropout=False,
82
+ init_type='normal', init_gain=0.02, gpu_ids=[])
83
+ sd = torch.load('net_G_A.pth', map_location='cpu')
84
+ if hasattr(sd, '_metadata'):
85
+ del sd._metadata
86
+ G_A.load_state_dict(sd)
87
+ G_A.eval()
88
+
89
+ sar = Image.open('sar.png').convert('RGB')
90
+ sar = sar.crop(((600-512)//2, (600-512)//2, (600+512)//2, (600+512)//2))
91
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
92
+
93
+ with torch.no_grad():
94
+ y = G_A(x) # this is fake_B
95
+
96
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
97
+ Image.fromarray(eo).save('eo.png')
98
+ ```
99
+
100
+ Upstream also supports a SAR-only run without this snippet:
101
+ `test.py --model test --dataset_mode single --no_dropout --model_suffix _A`,
102
+ which loads `latest_net_G_A.pth` directly.
103
+
104
+ ## ⚠ This cell fails the input-copy audit
105
+
106
+ We audit every cell for **identity collapse**: a model that has learned to pass
107
+ its input through rather than translate it. The test compares the mean absolute
108
+ distance from the generated image to the SAR **input** against the distance to
109
+ the EO **target**; a ratio below 1.0 means the output is closer to the input it
110
+ was given than to the answer it was asked for.
111
+
112
+ | quantity | value |
113
+ |---|---|
114
+ | mean \|generated − ground truth\| | 50.812 |
115
+ | mean \|generated − SAR input\| | 40.415 |
116
+ | **ratio** (SAR distance ÷ GT distance) | **0.795** — below 1.0 |
117
+ | verdict | **COLLAPSED** |
118
+ | cross-pair separation | 0.885 (cohort median 1.088) — no oracle behaviour |
119
+
120
+ **Read the metrics on this page with that in mind.** They are reported unchanged
121
+ because they are what the released implementation does at its own published
122
+ protocol, and a benchmark that quietly substitutes a better-behaved variant is
123
+ not reporting the method. But they should not be read as translation quality.
124
+ The separation figure is the companion check for the *opposite* failure: it is
125
+ the mean distance from a generated image to **other** images' ground truth
126
+ divided by the distance to its **own**, so a value well above the cohort median
127
+ would indicate that ground-truth information had leaked into the generation.
128
+ This cell sits at or below the median, so there is no leak — the problem here is
129
+ input-copying, and only input-copying.
130
+
131
+ This is not a budget artefact on this dataset: a diagnostic control run of the same code at roughly 2.2× the budget still collapses (ratio 0.767). That control is not published here.
132
+
133
+ ## Read before using this checkpoint
134
+
135
+ * **`net_G_A` is SAR → EO.** Loading `net_G_B` and expecting an EO image gives
136
+ you the reverse mapping and a confusing result.
137
+ * Build the network with `gpu_ids=[]` — the checkpoints have bare keys and a
138
+ non-empty `gpu_ids` introduces a `module.` prefix the load will reject.
139
+ * Normalisation is `[-1, 1]`.
140
+ * CycleGAN is **unpaired**: it never saw a SAR/EO correspondence during
141
+ training, only the two marginal distributions.
142
+
143
+ ## Licence — stated factually
144
+
145
+ Same upstream, same LICENSE file as pix2pix; reproduced verbatim here as
146
+ `licenses/LICENSE-cyclegan.txt` (byte-identical to `licenses/LICENSE-pix2pix.txt`
147
+ — one file covers both methods upstream). It is three stacked notices:
148
+
149
+ * CycleGAN — `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park` (BSD-2-Clause in
150
+ form);
151
+ * pix2pix — `Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu`;
152
+ * DCGAN — `Copyright (c) 2015, Facebook, Inc.` (BSD-3-Clause, with a
153
+ no-endorsement clause).
154
+
155
+ **Redistribution of these weights is permitted** with the notice reproduced. No
156
+ non-commercial and no share-alike clause. Do not use Facebook's name to promote
157
+ this release.
158
+
159
+ Attribution: *trained with `junyanz/pytorch-CycleGAN-and-pix2pix`. Copyright (c)
160
+ 2017, Jun-Yan Zhu and Taesung Park (CycleGAN); Copyright (c) 2016, Phillip Isola
161
+ and Jun-Yan Zhu (pix2pix); Copyright (c) 2015, Facebook, Inc. (DCGAN). All rights
162
+ reserved. Redistributed under the BSD terms reproduced in
163
+ `licenses/LICENSE-cyclegan.txt`.*
164
+
165
+ ---
166
+
167
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
168
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
169
+ itself. Every comparison method in this repository was retrained by us on the
170
+ same splits at the same resolution and scored through one evaluation pipeline,
171
+ so the rows are directly comparable to each other — and, for the same reason,
172
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/ddpm/README.md ADDED
@@ -0,0 +1,176 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # DDPM (SR3-class) — SAR2Opt
2
+
3
+ The **SR3 method class** — an eps-prediction conditional DDPM that conditions on
4
+ the source image — as introduced in *Image Super-Resolution via Iterative
5
+ Refinement*, Saharia, Ho, Chan, Salimans, Fleet and Norouzi,
6
+ *IEEE TPAMI 2022* ([arXiv:2104.07636](https://arxiv.org/abs/2104.07636)).
7
+
8
+ **Read this before citing the row: the code is not the SR3 authors'.** No code
9
+ from any SR3 release was used. This cell is
10
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) at commit
11
+ `38601093ab8f8e4b478144621f20890b100a3b74`, run in its **stage-1**
12
+ configuration (`"stage": 1`), which is an SR3-class conditional DDPM. We
13
+ retrained it ourselves on SAR2Opt and it is the checkpoint behind the DDPM
14
+ row of the ReFlowSET comparison table.
15
+
16
+ ## What is in this folder
17
+
18
+ | file | bytes | what it is |
19
+ |---|---|---|
20
+ | `gen.pth` | 768,426,944 | the `sr3` UNet at iteration 250,000 — the whole model |
21
+
22
+ `gen.pth` is E3Diff's **`sr3` UNet**: `inner_channel` 64,
23
+ `channel_multiplier` [1, 2, 4, 8, 16], `res_blocks` 1, `norm_groups` 16, **no
24
+ attention** (`attn_res` is empty), 3 input channels, 3 output channels and a
25
+ 3-channel condition.
26
+
27
+ The condition is **not the raw SAR image alone**. It is a three-channel stack
28
+ `[PPB, Canny, SAR]`:
29
+
30
+ * channel 0 — the SAR image after **FAST_PPB** speckle filtering
31
+ (Deledalle 2009, P = 3, W = 10, h = 0.5), a GPU port of E3Diff's own
32
+ `FAST_PPB.m`;
33
+ * channel 1 — `cv2.Canny(ppb_uint8, 50, 150, L2gradient=True)` on that filtered
34
+ image;
35
+ * channel 2 — the raw SAR image.
36
+
37
+ **This row cannot be run from a SAR PNG alone**: the PPB and Canny images must
38
+ be computed first.
39
+
40
+ ## Training budget we used
41
+
42
+ | | |
43
+ |---|---|
44
+ | **generator updates released** | **250,000** |
45
+ | batch size / resolution | 4 @ 512 px |
46
+ | optimizer | Adam, lr 5e-5, constant (`MultiStepLR` with no milestones) |
47
+ | EMA | decay 0.9999, started at step 5,000, updated every step |
48
+ | diffusion (train) | T = 1,000, linear β from 1e-6 to 1e-2 |
49
+ | sampler (test) | **DDIM, 50 steps** |
50
+ | loss | plain eps-prediction MSE — the stage-2 auxiliary losses are all zero here (`lpips_w` 0, `fft_w` 0, `lambda_gan` 0) |
51
+ | augmentation | flips, 90° rotations, and a random brightness jitter applied to the SAR/PPB pair only (upstream's own `transform_augment`) |
52
+
53
+ Both datasets get the same 250,000 updates; the batch differs so that the
54
+ pixels-per-update budget matches (16 × 256² = 4 × 512²).
55
+
56
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
57
+
58
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
59
+ |---|---|---|---|---|
60
+ | 122.5 | 0.295 | 0.610 | 0.313 | 13.65 |
61
+
62
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
63
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
64
+ method's training and in the evaluation. No subsampling: every metric on this
65
+ page is measured over all 627 pairs.
66
+
67
+ These are our own re-evaluation numbers, measured by us on the images this
68
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
69
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
70
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
71
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
72
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
73
+ convention gives a systematically different number and the two must never be
74
+ mixed, or compared against a paper that used the other one.
75
+
76
+ ## Load it and translate one SAR image
77
+
78
+ This model cannot be driven from a bare SAR file: it needs the PPB and Canny
79
+ condition channels, and the E3Diff loader looks them up by filename. Build the
80
+ condition first, then run the repository's own entry point.
81
+
82
+ ```python
83
+ # 1. Build the condition channels for each SAR image, once.
84
+ import cv2, numpy as np
85
+ # ppb = FAST_PPB(sar, P=3, W=10, h=0.5) # Deledalle 2009; E3Diff ships FAST_PPB.m
86
+ # canny = cv2.Canny(ppb.astype(np.uint8), 50, 150, L2gradient=True)
87
+ # Write ppb and canny under <root>/val/SAR-PPB/<name> and <root>/val/SAR-canny/<name>,
88
+ # alongside <root>/val/SAR/<name> and <root>/val/EO/<name>. Names must match exactly.
89
+
90
+ # 2. Run the repository, phase 'val', with a config whose
91
+ # path.resume_state is the checkpoint PREFIX -- no '_gen.pth' suffix:
92
+ # "path": {"resume_state": "<dir>/I250000_E<epoch>"} # place gen.pth as I250000_E<epoch>_gen.pth
93
+ # "stage": 1,
94
+ # "model": {"beta_schedule": {"val": {"n_timestep": 50, "ddim": 1, ...}}},
95
+ # "datasets": {"val": {"data_len": -1, "r_resolution": 512}}
96
+ # python main.py -c <val.json> -p val -enable_wandb "" --seed 1
97
+ ```
98
+
99
+ Two mechanical details that will otherwise cost you a full inference pass:
100
+
101
+ * the RGB loader. Upstream's `SAR2EODataset` hard-codes a **grayscale** EO
102
+ target and a two-channel `[PPB, Canny]` condition. SAR2Opt's optical side is
103
+ RGB, and with three channels upstream's `ddim_sample()` slices the condition to
104
+ `condition_x[:, :3]` and concatenates it with 3-channel predictions, which
105
+ raises on the first validation image. We ran through a small wrapper that
106
+ patches `SAR2EODataset.__getitem__` **in memory** — returning
107
+ `HR = EO[0:3]`, `LR = SAR[0:3]`, `SR = cat(PPB[0:1], Canny[0:1], SAR[0:1])` —
108
+ and then executes the repository's `main.py` verbatim, so no upstream file is
109
+ touched and every flag behaves the same. Reproduce that patch or write your own
110
+ three-channel loader;
111
+ * `-enable_wandb ""` is mandatory. The flag defaults to the string `'false'`,
112
+ which is truthy.
113
+
114
+ ## Read before using this checkpoint
115
+
116
+ * **The checkpoint loads with `strict=False`.** `model/model.py` calls
117
+ `load_state_dict(..., strict=False)`, so a configuration that does not match
118
+ the file **loads nothing and raises no error** — you get a fully random model
119
+ and a plausible-looking run. Verify the `Loading pretrained model for G [...]`
120
+ line in the log and eyeball the first output.
121
+ * **`-p val` writes next to the checkpoint and then renames.** Images land in
122
+ `<resume_state>/sample/`, and at the end the directory is renamed to
123
+ `<resume_state>_S<ssim>_P<psnr>_l2<l2>_Lp<lpips>`. That rename **raises if the
124
+ target already exists** — after the entire inference has been paid for. Guard
125
+ for an existing `<prefix>_S*/sample` before starting.
126
+ * **SoftPool.** The E3Diff code imports the SoftPool CUDA extension
127
+ unconditionally. We used a pure-PyTorch drop-in with identical maths
128
+ (`softpool(x) = avgpool(x·eˣ) / avgpool(eˣ)`) and identical autograd, which
129
+ needs no build step. Either build the extension or write the equivalent.
130
+ * **A known tone offset, inherited deliberately.** On another dataset in the
131
+ wider campaign this exact configuration produced generations that were, on
132
+ average, tens of gray levels brighter than the ground truth, at a substantial
133
+ cost in PSNR. We did **not** correct it per dataset, because comparability
134
+ across the row was judged worth more than per-cell tuning. Read this row's
135
+ PSNR with that in mind.
136
+ * Passes the identity-collapse audit on both datasets.
137
+
138
+ ## Licence — stated factually ⚠ no upstream licence exists
139
+
140
+ **The upstream code base publishes no licence.**
141
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff), whose stage-1
142
+ configuration this row is, has no LICENSE, LICENCE, COPYING or NOTICE file at
143
+ any depth in its tree; only 1 of its 58 tracked files is licence-shaped and that
144
+ one is `SoftPool/LICENSE.txt`, the MIT licence of a **vendored third-party
145
+ dependency** (`Copyright (c) 2020 Alexandros Stergiou`, reproduced here as
146
+ `licenses/LICENSE-SoftPool-MIT.txt`) — not a grant for E3Diff. Its README has no
147
+ licence section, and the GitHub API reports no declared licence, with the
148
+ `/license` endpoint returning 404. Checked 2026-08-28.
149
+
150
+ Under default copyright that means **all rights are reserved by the authors and
151
+ no express permission to redistribute derived work has been granted** to us or
152
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
153
+ end to end, and we state the position plainly rather than implying a permission
154
+ that does not exist. If you intend to redistribute this checkpoint or build on
155
+ it, assess that for yourself, and consider asking the authors directly.
156
+
157
+ The code base credits SR3
158
+ (`Janspiry/Image-Super-Resolution-via-Iterative-Refinement`, Apache-2.0),
159
+ `GaParmar/img2img-turbo` (MIT) and `alexandrosstergiou/SoftPool` (MIT). Those
160
+ licences cover the borrowed parts, not the authors' own contribution.
161
+
162
+ Please cite the SR3 paper for the method class, and E3Diff for the code that
163
+ produced these weights: Qin, Zou, Li and Zhang, *Efficient End-to-End Diffusion
164
+ Model for One-Step SAR-to-Optical Translation*, IEEE Geoscience and Remote
165
+ Sensing Letters, [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).
166
+ The full record of what we checked, per method, is in
167
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
168
+
169
+ ---
170
+
171
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
172
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
173
+ itself. Every comparison method in this repository was retrained by us on the
174
+ same splits at the same resolution and scored through one evaluation pipeline,
175
+ so the rows are directly comparable to each other — and, for the same reason,
176
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/e3diff/README.md ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # E3Diff — SAR2Opt
2
+
3
+ **Efficient End-to-End Diffusion Model for One-Step SAR-to-Optical
4
+ Translation**, Qin, Zou, Li and Zhang, *IEEE Geoscience and Remote Sensing
5
+ Letters*, 2024
6
+ ([doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566); the
7
+ year is the one in the authors' own citation block — a table that labels this row
8
+ with a later issue year is referring to the same paper). Upstream code:
9
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) at commit
10
+ `38601093ab8f8e4b478144621f20890b100a3b74`.
11
+
12
+ This is the authors' two-stage method, **retrained by us end to end** on
13
+ SAR2Opt, and it is the checkpoint behind the E3Diff row of the ReFlowSET
14
+ comparison table.
15
+
16
+ **These are our weights, not the authors' released checkpoint.** The authors'
17
+ own weights exist for a different dataset and are not what this row measures.
18
+
19
+ ## What is in this folder
20
+
21
+ | file | bytes | what it is |
22
+ |---|---|---|
23
+ | `gen.pth` | 768,426,944 | the `sr3` UNet after stage 2, at absolute iteration 310,000 |
24
+
25
+ Architecturally this is **the same network as the DDPM (SR3-class) row** in this
26
+ repository — E3Diff's two stages share one UNet. What differs is the training:
27
+
28
+ * **stage 1** (250,000 updates) is the eps-prediction conditional DDPM. Its
29
+ checkpoint is the `ddpm/` folder in this repository.
30
+ * **stage 2** (60,000 further updates, absolute iteration 310,000) fine-tunes
31
+ that same network into a **one-step generator**: the sampler is run *with
32
+ gradients* from pure noise for `ddim_steps = 1`, and the loss is taken directly
33
+ on pixels.
34
+
35
+ UNet: `inner_channel` 64, `channel_multiplier` [1, 2, 4, 8, 16], `res_blocks` 1,
36
+ `norm_groups` 16, no attention, 3 in / 3 out, 3-channel condition.
37
+
38
+ The condition is the same three-channel `[PPB, Canny, SAR]` stack described on
39
+ the DDPM card: FAST_PPB speckle filtering (Deledalle 2009, P = 3, W = 10,
40
+ h = 0.5), `cv2.Canny(ppb, 50, 150, L2gradient=True)`, and the raw SAR image.
41
+ **This row cannot be run from a SAR PNG alone.**
42
+
43
+ ## Training budget we used
44
+
45
+ | | |
46
+ |---|---|
47
+ | **generator updates released** | **250,000 (stage 1, inherited) + 60,000 (stage 2) = 310,000 absolute** |
48
+ | batch size / resolution | 4 @ 512 px |
49
+ | optimizer | Adam, lr 5e-5 |
50
+ | stage-2 sampler during training | DDIM, **1 step**, run with gradients from pure noise |
51
+ | stage-2 losses | L1 on pixels, + LPIPS (weight 5), + focal-frequency (weight 10), + a vision-aided CLIP GAN (λ_gan 0.5) |
52
+ | stage-1 losses | plain eps-prediction MSE (all auxiliary weights zero) |
53
+ | sampler (test) | **DDIM, 1 step** |
54
+ | EMA | decay 0.9999 (stage 1) |
55
+ | augmentation | flips, 90° rotations, and a random brightness jitter on the SAR/PPB pair (upstream's `transform_augment`) |
56
+
57
+ The stage-2 budget is 24 % of the stage-1 budget, which follows the authors' own
58
+ ratio (they resume at 640,000 and run to 800,000). Note that `n_iter` in this
59
+ code base is an **absolute** step count continuing stage 1, not a stage-2
60
+ budget — a config that reads 310,000 buys 60,000 new iterations.
61
+
62
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
63
+
64
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
65
+ |---|---|---|---|---|
66
+ | 104.7 | 0.232 | 0.529 | 0.249 | 16.09 |
67
+
68
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
69
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
70
+ method's training and in the evaluation. No subsampling: every metric on this
71
+ page is measured over all 627 pairs.
72
+
73
+ These are our own re-evaluation numbers, measured by us on the images this
74
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
75
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
76
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
77
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
78
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
79
+ convention gives a systematically different number and the two must never be
80
+ mixed, or compared against a paper that used the other one.
81
+
82
+ ## Load it and translate one SAR image
83
+
84
+ Same entry point and same mechanics as the DDPM row, with `"stage": 2` and
85
+ `ddim_steps: 1`.
86
+
87
+ ```python
88
+ # 1. Build the condition channels for each SAR image, once:
89
+ # ppb = FAST_PPB(sar, P=3, W=10, h=0.5)
90
+ # canny = cv2.Canny(ppb.astype('uint8'), 50, 150, L2gradient=True)
91
+ # written to <root>/val/SAR-PPB/<name> and <root>/val/SAR-canny/<name>,
92
+ # alongside <root>/val/SAR/<name> and <root>/val/EO/<name>.
93
+
94
+ # 2. Run phase 'val' with a config whose path.resume_state is the checkpoint
95
+ # PREFIX -- no '_gen.pth' suffix -- and place gen.pth as <prefix>_gen.pth:
96
+ # "stage": 2, "ddim_steps": 1,
97
+ # "model": {"beta_schedule": {"val": {"n_timestep": 1, "ddim": 1, ...}}},
98
+ # "datasets": {"val": {"data_len": -1, "r_resolution": 512}}
99
+ # python main.py -c <val.json> -p val -enable_wandb "" --seed 1
100
+ ```
101
+
102
+ Sampling is genuinely one network evaluation per image, which is the point of
103
+ the method.
104
+
105
+ As on the DDPM card, the RGB loader is the one thing you must sort out: upstream
106
+ hard-codes a grayscale EO target and a two-channel condition, and with three
107
+ channels its `ddim_sample()` slices the condition and concatenates mismatched
108
+ shapes. We patched `SAR2EODataset.__getitem__` in memory to return
109
+ `HR = EO[0:3]`, `LR = SAR[0:3]`, `SR = cat(PPB[0:1], Canny[0:1], SAR[0:1])` and
110
+ then ran the repository's `main.py` verbatim.
111
+
112
+ ## Read before using this checkpoint
113
+
114
+ * **This is not an oracle.** Worth stating, because a one-step generator invites
115
+ the question: the sampler's `condition_x` is the `[PPB, Canny, SAR]` stack, and
116
+ the ground truth never enters it. The cell also passes our leak audit on both
117
+ datasets.
118
+ * **Stage 2 needs the *real* vision-aided CLIP discriminator.** Stage 1 has
119
+ `lambda_gan = 0` and can import a stub; stage 2 cannot. If a constant-output
120
+ stub shadows the real `vision_aided_loss` package, stage 2 trains against a
121
+ constant GAN loss and **silently is not the authors' method**. Assert on
122
+ `vision_aided_loss.__file__` before starting a run.
123
+ * **SoftPool.** The code imports the SoftPool CUDA extension unconditionally. We
124
+ used a pure-PyTorch drop-in with identical maths
125
+ (`softpool(x) = avgpool(x·eˣ) / avgpool(eˣ)`) and identical autograd, which
126
+ needs no build step.
127
+ * **The checkpoint loads with `strict=False`.** A mismatched configuration
128
+ **loads nothing and raises no error**. Verify the `Loading pretrained model for
129
+ G [...]` log line and look at the first output.
130
+ * **`-p val` writes next to the checkpoint and then renames** the sample
131
+ directory to `<prefix>_S<ssim>_P<psnr>_l2<l2>_Lp<lpips>`. That rename raises if
132
+ the target already exists — after the whole inference has been paid for.
133
+ * **`-enable_wandb ""` is mandatory**; the flag defaults to the truthy string
134
+ `'false'`.
135
+ * One change we made to the repository: `core/logger.py` no longer
136
+ unconditionally overwrites `CUDA_VISIBLE_DEVICES` from the config's `gpu_ids`,
137
+ which on a shared machine remapped every job onto another user's device. It is
138
+ a scheduling fix and does not affect the model.
139
+
140
+ ## Licence — stated factually ⚠ no upstream licence exists
141
+
142
+ **The upstream code base publishes no licence.**
143
+ [`DeepSARRS/E3Diff`](https://github.com/DeepSARRS/E3Diff) has no LICENSE,
144
+ LICENCE, COPYING or NOTICE file at the repository root or at any depth; **1 of
145
+ its 58 tracked files** is licence-shaped, and it is `SoftPool/LICENSE.txt`, the
146
+ MIT licence of a **vendored third-party dependency** (`Copyright (c) 2020
147
+ Alexandros Stergiou`, reproduced here as `licenses/LICENSE-SoftPool-MIT.txt`) —
148
+ not a grant covering E3Diff. Its README has no licence section, and the GitHub
149
+ API reports no declared licence, with the `/license` endpoint returning 404.
150
+ Checked 2026-08-28.
151
+
152
+ Under default copyright that means **all rights are reserved by the authors and
153
+ no express permission to redistribute derived work has been granted** to us or
154
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
155
+ end to end, and we state the position plainly rather than implying a permission
156
+ that does not exist. If you intend to redistribute this checkpoint or build on
157
+ it, assess that for yourself, and consider asking the authors directly.
158
+
159
+ **Lineage.** E3Diff's README credits SR3
160
+ (`Janspiry/Image-Super-Resolution-via-Iterative-Refinement`, Apache-2.0),
161
+ `GaParmar/img2img-turbo` (MIT) and `alexandrosstergiou/SoftPool` (MIT). Those
162
+ licences cover the borrowed parts only, not the authors' own two-stage
163
+ contribution. We ship only the SoftPool text, because it is the only one of the
164
+ three that is vendored in the tree we trained from.
165
+
166
+ Please cite: Qin, Zou, Li and Zhang, *Efficient End-to-End Diffusion Model for
167
+ One-Step SAR-to-Optical Translation*, IEEE Geoscience and Remote Sensing
168
+ Letters, [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).
169
+ The full record of what we checked, per method, is in
170
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
171
+
172
+ ---
173
+
174
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
175
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
176
+ itself. Every comparison method in this repository was retrained by us on the
177
+ same splits at the same resolution and scored through one evaluation pipeline,
178
+ so the rows are directly comparable to each other — and, for the same reason,
179
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/hidiff/README.md ADDED
@@ -0,0 +1,164 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # HI-Diff — SAR2Opt
2
+
3
+ **Hierarchical Integration Diffusion Model for Realistic Image Deblurring**,
4
+ Chen, Zhang, Liu, Kong and Yuan, *NeurIPS 2023*
5
+ ([arXiv:2305.12966](https://arxiv.org/abs/2305.12966)). Upstream code:
6
+ [`zhengchen1999/HI-Diff`](https://github.com/zhengchen1999/HI-Diff) at commit
7
+ `b3bfd167997e27f8edd57681cf70e5031a0e35f2`.
8
+
9
+ This is the two-stage latent-prior restoration transformer we retrained
10
+ ourselves on SAR2Opt (SAR as the degraded input, EO as the target), and it is
11
+ the checkpoint behind the HI-Diff row of the ReFlowSET comparison table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `S1_net_g_latest.pth` | 101,401,457 | stage-1 restoration transformer — for reproducing stage-2 *training*, not for inference |
18
+ | `S1_net_le_latest.pth` | 2,357,509 | stage-1 latent encoder, `in_chans` **6** (it sees LQ ‖ GT) — **not usable at test time** |
19
+ | `S2_net_d_latest.pth` | 10,530,491 | **the 8-step latent denoiser** — needed at inference |
20
+ | `S2_net_g_latest.pth` | 101,401,457 | **the final restoration transformer** — needed at inference |
21
+ | `S2_net_le_dm_latest.pth` | 2,247,061 | **the stage-2 latent encoder**, `in_chans` 3 (LQ only) — needed at inference |
22
+
23
+ **Only the three `S2_*` files are needed to run this model.** The repository's
24
+ own test configuration loads exactly `pretrain_network_g`,
25
+ `pretrain_network_le_dm` and `pretrain_network_d`, and nothing else. The two
26
+ `S1_*` files are shipped so that stage-2 *training* can be reproduced: stage 1
27
+ learns a latent prior from the LQ **and** the ground truth together, which is why
28
+ its encoder takes 6 input channels and cannot be used at deployment.
29
+
30
+ Architecture: the transformer is Restormer-shaped — `dim` 48, blocks
31
+ [3, 5, 6, 6], 4 refinement blocks, heads [1, 2, 4, 8], FFN expansion 2.66,
32
+ `embed_dim` 64, `group` 4. The latent denoiser runs **8 timesteps** on the
33
+ compressed prior (`linear_start` 0.1, `linear_end` 0.99) — the diffusion here
34
+ acts on a small latent prior, not on the image.
35
+
36
+ ## Training budget we used
37
+
38
+ | | |
39
+ |---|---|
40
+ | **generator updates released** | **25,000 (stage 1) + 25,000 (stage 2) = 50,000** |
41
+ | batch size / patch | 8 @ 256 px |
42
+ | optimizer | AdamW, lr 2e-4, weight decay 1e-4, β (0.9, 0.999), gradient clipping on |
43
+ | schedule | `CosineAnnealingRestartCyclicLR`, periods [8,000 / 17,000], η_min [2e-4, 1e-6] |
44
+ | loss | L1 |
45
+ | progressive training | **disabled** |
46
+ | test resolution | trained on random 256 px crops and **tested at 512 px** — the network is fully convolutional, so the test resolution is not the training resolution for this cell |
47
+
48
+ The upstream recipe is 300,000 + 300,000 iterations with a progressive
49
+ patch/batch schedule from 128 to 384 px. We disabled the progressive schedule
50
+ (a single fixed 256 px group) and matched the budget to the rest of this
51
+ benchmark's diffusion band at 50,000 total updates. That is a substantial
52
+ reduction from the authors' recipe and is stated here so nobody reads this row
53
+ as HI-Diff at full budget.
54
+
55
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
56
+
57
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
58
+ |---|---|---|---|---|
59
+ | 319.8 | 0.473 | 0.692 | 0.384 | 17.36 |
60
+
61
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
62
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
63
+ method's training and in the evaluation. No subsampling: every metric on this
64
+ page is measured over all 627 pairs.
65
+
66
+ These are our own re-evaluation numbers, measured by us on the images this
67
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
68
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
69
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
70
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
71
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
72
+ convention gives a systematically different number and the two must never be
73
+ mixed, or compared against a paper that used the other one.
74
+
75
+ ## Load it and translate one SAR image
76
+
77
+ The three networks must be assembled by hand; the repository's own `test.py`
78
+ with a `HI_Diff_S2` option file is the path of least resistance.
79
+
80
+ ```yaml
81
+ # options/test/<your>.yml -- the three paths below are the three S2_ files
82
+ model_type: HI_Diff_S2
83
+ network_g: {type: Transformer, inp_channels: 3, out_channels: 3, dim: 48,
84
+ num_blocks: [3,5,6,6], num_refinement_blocks: 4, heads: [1,2,4,8],
85
+ ffn_expansion_factor: 2.66, bias: False, LayerNorm_type: WithBias,
86
+ dual_pixel_task: False, embed_dim: 64, group: 4}
87
+ network_le_dm: {type: latent_encoder_gelu, in_chans: 3, embed_dim: 64, block_num: 6,
88
+ group: 4, stage: 2, patch_expansion: 0.5, channel_expansion: 4}
89
+ network_d: {type: denoising, in_channel: 256, out_channel: 256, inner_channel: 512,
90
+ block_num: 4, group: 4, patch_expansion: 0.5, channel_expansion: 2}
91
+ diffusion_schedule: {apply_ldm: False, schedule: linear, timesteps: 8,
92
+ linear_start: 0.1, linear_end: 0.99}
93
+ path:
94
+ pretrain_network_g: S2_net_g_latest.pth
95
+ pretrain_network_le_dm: S2_net_le_dm_latest.pth
96
+ pretrain_network_d: S2_net_d_latest.pth
97
+ param_key_g: params
98
+ ```
99
+
100
+ ```bash
101
+ python test.py -opt options/test/<your>.yml
102
+ ```
103
+
104
+ The SAR input is read with `cv2.IMREAD_COLOR`, so a single-channel SAR image is
105
+ replicated to three channels automatically — that is what training saw. At
106
+ inference `le_dm` produces the prior from the degraded input alone, `net_d`
107
+ denoises that prior over 8 steps, and `net_g` is conditioned on it.
108
+
109
+ ## Read before using this checkpoint
110
+
111
+ * **The `S1_*` files are not an inference path.** `S1_net_le_latest.pth` has
112
+ `in_chans: 6` because stage 1 encodes LQ concatenated with the ground truth.
113
+ Wiring it into a test run would make the model an oracle. Use the `S2_*` trio.
114
+ * **These are the `latest` checkpoints — iteration 25,000 — not a best-validation
115
+ selection.** That is deliberate and benchmark-wide: every other method here
116
+ publishes its last checkpoint, so selecting on validation for this one would
117
+ be an asymmetry in its favour. Do not repoint at a best-val file and then
118
+ compare against this table.
119
+ * **Retraining is destructive.** The upstream launcher path is not idempotent:
120
+ the training configs set `resume_state: ~`, no auto-resume flag is passed, and
121
+ basicsr's `mkdir_and_rename` archives a finished experiment directory and
122
+ starts from scratch. Re-running a training script over a finished experiment
123
+ destroys it.
124
+ * **`torch.load` needs `weights_only=False`** under PyTorch ≥ 2.6; we made that
125
+ change in two places in the repository (`hi_diff/utils/base_model.py` and
126
+ `train.py`). Both are compatibility fixes.
127
+ * **How to read this row's numbers.** HI-Diff takes the best PSNR and SSIM of
128
+ any method in this benchmark on both datasets, while sitting at or near the
129
+ worst FID, DISTS and LPIPS. That combination is the signature of
130
+ regression-to-the-mean: a blurred, low-variance prediction minimises squared
131
+ error and is perceptually poor. Reporting its PSNR without its FID would badly
132
+ misrepresent it — and so would the reverse.
133
+ * Passes the identity-collapse audit on both datasets.
134
+
135
+ ## Licence — stated factually
136
+
137
+ Upstream `zhengchen1999/HI-Diff` ships a LICENSE file, reproduced verbatim here
138
+ as `licenses/LICENSE-hidiff.txt`: the **Apache License 2.0**, appendix copyright
139
+ line `Copyright 2023 HI-Diff Authors`.
140
+
141
+ **Redistribution of these weights is permitted.** Apache-2.0 §4 obligations
142
+ apply: ship the licence, keep the notices, and **state that you modified the
143
+ files**. We did modify them — the two `torch.load(..., weights_only=False)`
144
+ changes noted above, plus our own option files — and this sentence is that
145
+ statement. Upstream ships no NOTICE file, so there is none to carry. Apache-2.0
146
+ §6 withholds trademark rights.
147
+
148
+ HI-Diff itself builds on BasicSR, Restormer and DiffIR, which its README
149
+ credits; those projects' own licences cover their code.
150
+
151
+ Attribution: *trained with `zhengchen1999/HI-Diff`, Copyright 2023 HI-Diff
152
+ Authors, licensed under the Apache License 2.0; our training and test
153
+ configurations modify the upstream configuration, and two source files were
154
+ modified for PyTorch ≥ 2.6 compatibility. HI-Diff builds on BasicSR, Restormer
155
+ and DiffIR. Full text in `licenses/LICENSE-hidiff.txt`.*
156
+
157
+ ---
158
+
159
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
160
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
161
+ itself. Every comparison method in this repository was retrained by us on the
162
+ same splits at the same resolution and scored through one evaluation pipeline,
163
+ so the rows are directly comparable to each other — and, for the same reason,
164
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/p2phd/README.md ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # pix2pixHD — SAR2Opt
2
+
3
+ **High-Resolution Image Synthesis and Semantic Manipulation with Conditional
4
+ GANs**, Wang, Liu, Zhu, Tao, Kautz and Catanzaro, *CVPR 2018*
5
+ ([arXiv:1711.11585](https://arxiv.org/abs/1711.11585)). Upstream code:
6
+ [`NVIDIA/pix2pixHD`](https://github.com/NVIDIA/pix2pixHD) at commit
7
+ `14b3b3c7fff413086e3b58df52096f16b6891172`.
8
+
9
+ This is the coarse-to-fine paired GAN we retrained ourselves on SAR2Opt, and
10
+ it is the checkpoint behind the pix2pixHD row of the ReFlowSET comparison
11
+ table.
12
+
13
+ ## What is in this folder
14
+
15
+ | file | bytes | what it is |
16
+ |---|---|---|
17
+ | `net_G.pth` | 729,795,505 | the global generator, and the only file inference needs |
18
+
19
+ `net_G.pth` is the **`global` generator** (`GlobalGenerator`):
20
+ `n_downsample_global` 4, `n_blocks_global` 9 residual blocks, `ngf = 64`,
21
+ instance normalisation, 3 in / 3 out. The run was configured with
22
+ `n_local_enhancers 1`, but at `netG=global` the local enhancer is never
23
+ constructed and is not in this file — this is the single-scale generator.
24
+
25
+ The run used `--label_nc 0 --no_instance`, i.e. the "label" input is a real
26
+ image (the SAR chip) rather than a segmentation map, and no instance-edge
27
+ channel is concatenated. The two discriminators (`num_D` 2) are not released.
28
+
29
+ ## Training budget we used
30
+
31
+ | | |
32
+ |---|---|
33
+ | **generator updates released** | **36,200** |
34
+ | batch size | 8 |
35
+ | epochs (`niter` + `niter_decay`) | 100 + 100 |
36
+ | iterations per epoch | 181 (1,450 training pairs; this loader floor-rounds, unlike the junyanz loaders which round up) |
37
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay over the second half |
38
+ | adversarial loss | LSGAN, multi-scale discriminator (`num_D` 2, `n_layers_D` 3) |
39
+ | feature-matching loss | λ_feat 10 |
40
+ | perceptual loss | VGG loss enabled |
41
+ | augmentation | horizontal flip |
42
+ | input | random 512 crops of the 600 px tiles (`crop`, `loadSize` 600, `fineSize` 512) |
43
+
44
+ The floor-vs-ceiling detail is not pedantry: pix2pixHD and SPADE drop the last
45
+ partial batch while the junyanz-family loaders do not, so the same "epochs"
46
+ figure yields slightly different update counts across the table. The update
47
+ counts above are what actually ran.
48
+
49
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
50
+
51
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
52
+ |---|---|---|---|---|
53
+ | 146.3 | 0.283 | 0.567 | 0.268 | 15.95 |
54
+
55
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
56
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
57
+ method's training and in the evaluation. No subsampling: every metric on this
58
+ page is measured over all 627 pairs.
59
+
60
+ These are our own re-evaluation numbers, measured by us on the images this
61
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
62
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
63
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
64
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
65
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
66
+ convention gives a systematically different number and the two must never be
67
+ mixed, or compared against a paper that used the other one.
68
+
69
+ ## Load it and translate one SAR image
70
+
71
+ `GlobalGenerator` can be constructed directly, without the repository's option
72
+ machinery:
73
+
74
+ ```python
75
+ import numpy as np, torch, torch.nn as nn
76
+ from PIL import Image
77
+ from models.networks import GlobalGenerator # from the NVIDIA/pix2pixHD repo
78
+
79
+ G = GlobalGenerator(3, 3, 64, 4, 9, nn.InstanceNorm2d)
80
+ G.load_state_dict(torch.load('net_G.pth', map_location='cpu'))
81
+ G.eval()
82
+
83
+ sar = Image.open('sar.png').convert('RGB')
84
+ sar = sar.crop(((600-512)//2, (600-512)//2, (600+512)//2, (600+512)//2))
85
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
86
+
87
+ with torch.no_grad():
88
+ y = G(x)
89
+
90
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
91
+ Image.fromarray(eo).save('eo.png')
92
+ ```
93
+
94
+ Through the repository's own `test.py` the equivalent invocation is
95
+ `--label_nc 0 --no_instance --which_epoch latest`, and the output lands as
96
+ `<stem>_synthesized_image.png`.
97
+
98
+ ## Read before using this checkpoint
99
+
100
+ * **Upstream writes its test outputs as JPEG.** `util/visualizer.py` hard-codes
101
+ `'%s_%s.jpg'`. We changed that one line to `.png` before scoring, because
102
+ every other method in this benchmark writes PNG and JPEG re-compression would
103
+ have biased FID and LPIPS for this row alone. **If you clone upstream
104
+ unmodified and evaluate, your numbers will not match ours**, and the gap is an
105
+ artefact of the file format, not of the model.
106
+ * Three further changes we made to the repository are pure Python-3.12 /
107
+ modern-library compatibility and do not affect the weights:
108
+ `transforms.Scale` → `transforms.Resize`; `fractions.gcd` → `math.gcd` with
109
+ integer division; and a guarded `import scipy.misc` (needed only for
110
+ `--tf_log`, which we never pass).
111
+ * Normalisation is `[-1, 1]`.
112
+ * Passes the identity-collapse audit on both datasets.
113
+
114
+ ## Licence — stated factually
115
+
116
+ Upstream `NVIDIA/pix2pixHD` ships `LICENSE.txt`, reproduced verbatim here as
117
+ `licenses/LICENSE-p2phd.txt`. It contains two notices:
118
+
119
+ * `Copyright (C) 2019 NVIDIA Corporation. Ting-Chun Wang, Ming-Yu Liu,
120
+ Jun-Yan Zhu. BSD License. All rights reserved.` — two conditions and a
121
+ disclaimer;
122
+ * the bundled `pytorch-CycleGAN-and-pix2pix` notice,
123
+ `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park`, because pix2pixHD's code
124
+ borrows heavily from it.
125
+
126
+ **Redistribution of these weights is permitted** with both notices reproduced.
127
+ Despite NVIDIA's non-commercial licensing of some of its other releases, this
128
+ one carries **no non-commercial clause and no share-alike clause** — we checked
129
+ the file and the README, and the README has no licence section at all, so
130
+ `LICENSE.txt` is the entire grant.
131
+
132
+ Attribution: *trained with `NVIDIA/pix2pixHD`. Copyright (C) 2019 NVIDIA
133
+ Corporation — Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu. BSD License, all rights
134
+ reserved; includes the bundled pytorch-CycleGAN-and-pix2pix notice (Copyright
135
+ (c) 2017, Jun-Yan Zhu and Taesung Park). Full text in
136
+ `licenses/LICENSE-p2phd.txt`.*
137
+
138
+ ---
139
+
140
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
141
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
142
+ itself. Every comparison method in this repository was retrained by us on the
143
+ same splits at the same resolution and scored through one evaluation pipeline,
144
+ so the rows are directly comparable to each other — and, for the same reason,
145
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/pix2pix/README.md ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # pix2pix — SAR2Opt
2
+
3
+ **Image-to-Image Translation with Conditional Adversarial Networks**, Isola, Zhu,
4
+ Zhou and Efros, *CVPR 2017* ([arXiv:1611.07004](https://arxiv.org/abs/1611.07004)).
5
+ Upstream code:
6
+ [`junyanz/pytorch-CycleGAN-and-pix2pix`](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
7
+ at commit `2a7afba2895d52556dd5dfe07e8555ef657ced6f`.
8
+
9
+ This is the paired conditional GAN we retrained ourselves on SAR2Opt, and it
10
+ is the checkpoint behind the pix2pix row of the ReFlowSET comparison table.
11
+
12
+ ## What is in this folder
13
+
14
+ | file | bytes | what it is |
15
+ |---|---|---|
16
+ | `net_G.pth` | 217,728,293 | the generator, and the only file inference needs |
17
+
18
+ `net_G.pth` is a **`unet_256` generator**: a U-Net with batch normalisation,
19
+ `ngf = 64`, 3 input channels and 3 output channels, saved from a single-GPU run
20
+ so the state-dict keys are bare (no `module.` prefix). It maps **SAR → EO**:
21
+ the training set was built as side-by-side `A|B` tiles with SAR on the left, and
22
+ the run used `--direction AtoB`.
23
+
24
+ The discriminator is not part of this release — upstream's `test.py` constructs
25
+ only the generator, so `net_G.pth` is sufficient to reproduce every image we
26
+ scored.
27
+
28
+ ## Training budget we used
29
+
30
+ | | |
31
+ |---|---|
32
+ | **generator updates released** | **36,400** |
33
+ | batch size | 8 |
34
+ | epochs (constant LR + linear decay) | 100 + 100 |
35
+ | iterations per epoch | 182 (1,450 training pairs, the loader does not drop the last batch) |
36
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
37
+ | adversarial loss | vanilla GAN; discriminator `basic` (70×70 PatchGAN, `n_layers_D` 3) |
38
+ | reconstruction loss | L1, weight 100 |
39
+ | augmentation | horizontal flip |
40
+ | input | random 512 crops of the 600 px tiles (`resize_and_crop`, `load_size` 600, `crop_size` 512) |
41
+
42
+ We quote budgets in **optimizer updates**, never in epochs: the two datasets
43
+ differ by more than 10× in size, so "200 epochs" means something completely
44
+ different on each and would silently favour one of them.
45
+
46
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
47
+
48
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
49
+ |---|---|---|---|---|
50
+ | 261.9 | 0.347 | 0.657 | 0.199 | 13.39 |
51
+
52
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
53
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
54
+ method's training and in the evaluation. No subsampling: every metric on this
55
+ page is measured over all 627 pairs.
56
+
57
+ These are our own re-evaluation numbers, measured by us on the images this
58
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
59
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
60
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
61
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
62
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
63
+ convention gives a systematically different number and the two must never be
64
+ mixed, or compared against a paper that used the other one.
65
+
66
+ ## Load it and translate one SAR image
67
+
68
+ The snippet uses the upstream repository's own `define_G`, so clone
69
+ `junyanz/pytorch-CycleGAN-and-pix2pix` at the commit above and run this from its
70
+ root. No patch is needed for this method — our only change to that repository was
71
+ an import fix in the dataset-preparation helper, which does not affect the model.
72
+
73
+ ```python
74
+ import numpy as np, torch
75
+ from PIL import Image
76
+ from models.networks import define_G # from the junyanz repo
77
+
78
+ G = define_G(3, 3, 64, 'unet_256', 'batch', use_dropout=False,
79
+ init_type='normal', init_gain=0.02, gpu_ids=[])
80
+ sd = torch.load('net_G.pth', map_location='cpu')
81
+ if hasattr(sd, '_metadata'):
82
+ del sd._metadata # upstream strips this too
83
+ G.load_state_dict(sd)
84
+ G.eval()
85
+
86
+ # SAR input -> RGB -> [-1, 1]. SAR2Opt SAR is single-channel; .convert('RGB')
87
+ # replicates it to three channels, which is what training saw.
88
+ sar = Image.open('sar.png').convert('RGB')
89
+ sar = sar.crop(((600-512)//2, (600-512)//2, (600+512)//2, (600+512)//2))
90
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
91
+
92
+ with torch.no_grad():
93
+ y = G(x)
94
+
95
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
96
+ Image.fromarray(eo).save('eo.png')
97
+ ```
98
+
99
+ The image this produces is `<stem>_fake_B.png` in upstream's own `test.py`
100
+ output tree, which is exactly what we scored.
101
+
102
+ ## Read before using this checkpoint
103
+
104
+ * **`define_G` must be called with `gpu_ids=[]`.** Our checkpoints were saved
105
+ from single-GPU runs and have bare keys; building the network with a non-empty
106
+ `gpu_ids` wraps it in `DataParallel` and the load then fails on a `module.`
107
+ prefix mismatch.
108
+ * **The normalisation is `[-1, 1]`, not `[0, 1]`.** Feeding `[0, 1]` produces a
109
+ plausible-looking but systematically wrong image.
110
+ * This row is a **paired** method and was trained on aligned SAR/EO pairs. It
111
+ passes the identity-collapse audit on both datasets.
112
+
113
+ ## Licence — stated factually
114
+
115
+ Upstream `junyanz/pytorch-CycleGAN-and-pix2pix` ships a LICENSE file, reproduced
116
+ verbatim in this repository as `licenses/LICENSE-pix2pix.txt`. It is **not a
117
+ single SPDX licence** but three stacked notices, and the whole file is what
118
+ travels with these weights:
119
+
120
+ * CycleGAN — `Copyright (c) 2017, Jun-Yan Zhu and Taesung Park`, two conditions
121
+ and a disclaimer (BSD-2-Clause in form);
122
+ * pix2pix — `Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu`, two conditions;
123
+ * DCGAN — `Copyright (c) 2015, Facebook, Inc.`, three conditions including a
124
+ no-endorsement clause (BSD-3-Clause).
125
+
126
+ **Redistribution of these weights is permitted** with that notice reproduced.
127
+ There is no non-commercial and no share-alike clause. The DCGAN section's
128
+ no-endorsement clause means Facebook's name must not be used to promote this
129
+ release.
130
+
131
+ Attribution: *trained with `junyanz/pytorch-CycleGAN-and-pix2pix`. Copyright (c)
132
+ 2017, Jun-Yan Zhu and Taesung Park (CycleGAN); Copyright (c) 2016, Phillip Isola
133
+ and Jun-Yan Zhu (pix2pix); Copyright (c) 2015, Facebook, Inc. (DCGAN). All rights
134
+ reserved. Redistributed under the BSD terms reproduced in
135
+ `licenses/LICENSE-pix2pix.txt`.*
136
+
137
+ ---
138
+
139
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
140
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
141
+ itself. Every comparison method in this repository was retrained by us on the
142
+ same splits at the same resolution and scored through one evaluation pipeline,
143
+ so the rows are directly comparable to each other — and, for the same reason,
144
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/resshift/README.md ADDED
@@ -0,0 +1,146 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ResShift — SAR2Opt
2
+
3
+ **ResShift: Efficient Diffusion Model for Image Super-Resolution by Residual
4
+ Shifting**, Yue, Wang and Loy, *NeurIPS 2023*
5
+ ([arXiv:2307.12348](https://arxiv.org/abs/2307.12348)). Upstream code:
6
+ [`zsyOAOA/ResShift`](https://github.com/zsyOAOA/ResShift) at commit
7
+ `bb03b7d21614cace01787e097c8a6ab6b945227d`.
8
+
9
+ This is the residual-shifting latent diffusion model we retrained ourselves on
10
+ SAR2Opt, run as a **same-size** mapping (`sf = 1`: SAR in, EO out, no
11
+ upscaling), and it is the checkpoint behind the ResShift row of the ReFlowSET
12
+ comparison table.
13
+
14
+ ## What is in this folder
15
+
16
+ | file | bytes | what it is |
17
+ |---|---|---|
18
+ | `ema_model.pth` | 478,392,663 | EMA weights of the denoising UNet at iteration 50,000 |
19
+
20
+ `ema_model.pth` is the EMA (rate 0.999) of a **`models.unet.UNetModelSwin`**:
21
+ latent resolution 64, `model_channels` 160, `channel_mult` [1, 2, 2, 4],
22
+ `num_res_blocks` [2, 2, 2, 2], attention at [64, 32, 16, 8], 32 channels per
23
+ head, Swin blocks of depth 2 with `swin_embed_dim` 192 and window size 8,
24
+ `cond_lq = True`, `lq_size = 256`.
25
+
26
+ **This is a latent-space model and this file is not sufficient to run it.** It
27
+ additionally needs the external CompVis **vq-f4** autoencoder
28
+ (`ldm.models.autoencoder.VQModelTorch`, 221,364,711 bytes), which is **not**
29
+ redistributed here.
30
+
31
+ ## Training budget we used
32
+
33
+ | | |
34
+ |---|---|
35
+ | **generator updates released** | **50,000** |
36
+ | batch size | 16 (micro-batch 8, i.e. 2 gradient-accumulation steps per update) |
37
+ | optimizer | Adam, lr 5e-5 → 2e-5 cosine, 2,000-step warmup, no weight decay |
38
+ | EMA | rate 0.999 |
39
+ | diffusion | **4 steps**, exponential noise schedule (power 0.3), `etas_end` 0.99, `min_noise_level` 0.2, κ = 2.0, `predict_type` xstart, `sf` 1 |
40
+ | loss weights | latent MSE 1.0, pixel LPIPS (AlexNet) 4.0, pixel MSE 0.0 |
41
+ | precision | AMP on |
42
+ | training crops | random 256 px, normalised with mean 0.5 / std 0.5 |
43
+ | seed | 123456 |
44
+
45
+ ResShift is a **4-step** sampler by construction; that is the method, not a
46
+ shortcut we took.
47
+
48
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
49
+
50
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
51
+ |---|---|---|---|---|
52
+ | 141.7 | 0.304 | 0.597 | 0.177 | 14.31 |
53
+
54
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
55
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
56
+ method's training and in the evaluation. No subsampling: every metric on this
57
+ page is measured over all 627 pairs.
58
+
59
+ These are our own re-evaluation numbers, measured by us on the images this
60
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
61
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
62
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
63
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
64
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
65
+ convention gives a systematically different number and the two must never be
66
+ mixed, or compared against a paper that used the other one.
67
+
68
+ ## Load it and translate one SAR image
69
+
70
+ Upstream's `inference_resshift.py` is a super-resolution CLI keyed to the
71
+ authors' own released tasks and does not accept a SAR→EO configuration. We wrote
72
+ a small driver against the repository's `ResShiftSampler`; the equivalent is:
73
+
74
+ ```python
75
+ from omegaconf import OmegaConf
76
+ from sampler import ResShiftSampler # from the zsyOAOA/ResShift repo
77
+
78
+ cfg = OmegaConf.load('<your>_sar2eo_256.yaml') # sf 1, steps 4, predict_type xstart
79
+ cfg.model.ckpt_path = 'ema_model.pth'
80
+ cfg.autoencoder.ckpt_path = '<path to>/autoencoder_vq_f4.pth' # NOT shipped here
81
+
82
+ sampler = ResShiftSampler(cfg, sf=1, chop_size=256, chop_stride=256,
83
+ chop_bs=1, use_amp=True, seed=12345, padding_offset=None)
84
+ sampler.inference('<dir of SAR images>', '<output dir>', bs=8, noise_repeat=False)
85
+ ```
86
+
87
+ At 512 px each input is processed as four clean 256 px tiles (`chop_size = chop_stride = lq_size = 256`, so the tiles do not overlap). Outputs keep the input file stems.
88
+
89
+ ## Read before using this checkpoint
90
+
91
+ * **The external vq-f4 autoencoder is mandatory and is not in this repository.**
92
+ Without it the model cannot be constructed, let alone run. It is CompVis
93
+ latent-diffusion material under its own (MIT) licence.
94
+ * **Upstream's own inference CLI will not drive these weights.** It is written
95
+ around the authors' released super-resolution tasks. Use the sampler directly,
96
+ as above.
97
+ * **xFormers must be disabled.** On the accelerator we trained on, the installed
98
+ xFormers build has no usable kernel for these shapes (attention head dimension
99
+ 512 exceeds its limit, and fp32 is unsupported), so we set
100
+ `XFORMERS_IS_AVAILBLE = False` in three files
101
+ (`ldm/modules/attention.py`, `ldm/modules/diffusionmodules/model.py`,
102
+ `models/unet.py`). If your hardware differs, this is a performance switch, not
103
+ a correctness one.
104
+ * **One numerical guard in the image logger.** fp16 diffused latents decoded
105
+ through the fp32 VQGAN produce NaN/Inf at early iterations and crashed training
106
+ *inside the logging path*. We added a `nan_to_num` + `clamp` before
107
+ `make_grid`. It is logging-only; the loss path is untouched. (A separate
108
+ `nan_to_num` on the LPIPS loss is upstream's, not ours.)
109
+ * **There is no auto-resume.** A crash mid-run needs a manual relaunch pointing
110
+ at the last saved checkpoint.
111
+ * Passes the identity-collapse audit on both datasets.
112
+
113
+ ## Licence — stated factually ⚠ non-commercial
114
+
115
+ Upstream `zsyOAOA/ResShift` ships a LICENSE file, reproduced verbatim here as
116
+ `licenses/LICENSE-resshift.txt`: the **S-Lab License 1.0**,
117
+ `Copyright 2022 S-Lab`.
118
+
119
+ **This licence is non-commercial.** Its opening clause reads *"Redistribution
120
+ and use **for non-commercial purpose** in source and binary forms, with or
121
+ without modification, are permitted provided that the following conditions are
122
+ met"*, and it adds that commercial use requires contacting the contributors. The
123
+ README repeats it: *"This project is licensed under NTU S-Lab License 1.0.
124
+ Redistribution and use should follow this license."*
125
+
126
+ So: **redistribution of these weights is permitted for non-commercial purposes
127
+ only**, in binary form, with the copyright notice, the conditions and the
128
+ disclaimer reproduced — which is what `licenses/LICENSE-resshift.txt` is for.
129
+ Clause 3 is a **no-endorsement** clause: neither S-Lab's name nor its
130
+ contributors' names may be used to endorse or promote this release.
131
+
132
+ Attribution: *trained with `zsyOAOA/ResShift`, Copyright 2022 S-Lab, **S-Lab
133
+ License 1.0 — redistribution and use for non-commercial purposes only**; for
134
+ commercial use, contact the contributors. Requires the CompVis vq-f4
135
+ autoencoder (MIT), not redistributed here. Neither S-Lab's name nor its
136
+ contributors' names may be used to endorse this release. Full text in
137
+ `licenses/LICENSE-resshift.txt`.*
138
+
139
+ ---
140
+
141
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
142
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
143
+ itself. Every comparison method in this repository was retrained by us on the
144
+ same splits at the same resolution and scored through one evaluation pipeline,
145
+ so the rows are directly comparable to each other — and, for the same reason,
146
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/sd21ft/README.md ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SD2.1 fine-tune only — SAR2Opt
2
+
3
+ The **"SD2.1 fine-tune only"** ablation row of the ReFlowSET comparison table:
4
+ Stable Diffusion 2.1-base fine-tuned for SAR-to-EO translation with a plain MSE
5
+ epsilon loss and **no confidence head**.
6
+
7
+ > **This is not a stock SD 2.1 UNet and it will not load into a stock SD 2.1
8
+ > pipeline.** It is C-DiffSET **stage 1**: the input convolution is widened to
9
+ > 8 channels so the SAR latent can be concatenated to the noisy EO latent. Only
10
+ > the output side is unmodified.
11
+
12
+ Venue: there is no separate paper for this row. The backbone is Stable
13
+ Diffusion 2.1-base (latent diffusion, Rombach et al., CVPR 2022); the trainer
14
+ is the stage-1 trainer of
15
+ [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET)
16
+ (*IEEE TCSVT* 2026, [doi:10.1109/TCSVT.2026.3701447](https://doi.org/10.1109/TCSVT.2026.3701447)).
17
+
18
+ ## What is in this folder
19
+
20
+ A single `UNet2DConditionModel` (865,922,244 parameters, fp32, 686 tensors):
21
+
22
+ * **`conv_in` 4 -> 8 channels**, exactly as in the C-DiffSET folder — the input
23
+ is `torch.cat([sar_latent, noisy_eo_latent], dim=1)`, **SAR first**. In this
24
+ trained checkpoint the two halves have diverged (mean |w| 0.01545 on channels
25
+ 0-3 vs 0.02596 on channels 4-7).
26
+ * **`conv_out` unchanged at 4 channels.** No variance head, no confidence map.
27
+ The stage-1 trainer explicitly overrides the channel surgery to a no-op, and
28
+ the whole 4-channel prediction goes to the scheduler.
29
+
30
+ That 5-vs-4 output channel count is the only architectural difference from the
31
+ `cdiffset` folder in this repository; the parameter counts differ by exactly
32
+ 2,881 (one 320x3x3 filter plus one bias).
33
+
34
+ ## Training budget we used
35
+
36
+ | | |
37
+ |---|---|
38
+ | optimizer updates released | **40,000** (fixed-step snapshot `checkpoint-40000`) |
39
+ | updates the run was configured for | 50,000 |
40
+ | initialised from | the SD 2.1-base UNet with the duplicated-and-halved 8-channel `conv_in` |
41
+ | batch size / resolution | 16 @ 512 px |
42
+ | epochs the 40,000 updates work out to | ~444 (1,450 training pairs, 90 updates/epoch) |
43
+ | optimizer | AdamW, lr 3e-5, weight decay 0.01, cosine schedule, 100-step warmup |
44
+ | precision | fp32 (no mixed precision) |
45
+ | seed | 2024 |
46
+ | loss | plain MSE on the epsilon prediction |
47
+ | prompt | the fixed string `"electro-optical image"` |
48
+ | augmentation | horizontal flip, vertical flip, 90-degree rotation |
49
+ | test-time sampler | **DDIM, 50 steps**, fp32, one image at a time |
50
+
51
+ Same two caveats as the C-DiffSET folder: the snapshot is taken at update 40,000
52
+ of a cosine schedule laid out for 50,000, so the learning rate had not finished
53
+ decaying; and it is a fixed-step snapshot, not the repository's
54
+ validation-selected `best/`. For this stage the fixed-step choice is also the
55
+ better one on the merits — validation LPIPS bottoms out near update 41,000 and
56
+ then degrades toward 50,000.
57
+
58
+ Data: 512 px. The source images are 600 px and are **never resized**: training takes a random 512 crop, evaluation takes the centre 512 crop (offset 44). The EO/SAR pairing follows the official SAR2Opt layout (`trainB`/`testB` are EO, `trainA`/`testA` are SAR).
59
+
60
+ ## Base model
61
+
62
+ This checkpoint is a derivative of **Stable Diffusion 2.1-base**. Because the
63
+ original `stabilityai/stable-diffusion-2-1-base` repository is no longer
64
+ available on the Hub, our training and every snippet below load the community
65
+ mirror [`Manojb/stable-diffusion-2-1-base`](https://huggingface.co/Manojb/stable-diffusion-2-1-base).
66
+ That is what we actually trained against — verified in the training
67
+ configuration files, the launch scripts and the training logs, none of which
68
+ ever reference `stabilityai/...`. The frozen VAE, CLIP text encoder, tokenizer
69
+ and scheduler all come from that repo. A different SD 2.1-base mirror is
70
+ expected to work but has not been verified by us.
71
+
72
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
73
+
74
+ | FID↓ | KID↓ | DISTS↓ | LPIPS↓ | PSNR↑ | SSIM↑ |
75
+ |---|---|---|---|---|---|
76
+ | 71.8 | 0.0090 | 0.211 | 0.541 | 16.24 | 0.293 |
77
+
78
+ These are our own re-evaluation numbers, not numbers copied from any paper.
79
+ LPIPS is **LPIPS-VGG on inputs mapped to [-1, 1]** (the `normalize=False`
80
+ convention); the alternative [0, 1] convention gives a systematically different
81
+ number and the two must never be mixed. PSNR/SSIM are per-image torchmetrics
82
+ with `data_range=1`; FID is `pytorch-fid` against the size-matched ground truth.
83
+
84
+ > **DISTS on this cell was re-measured.** This cell's prediction dump was regenerated after the only extended-metric pass that scored it, and that pass resumes on the *number* of images rather than their content, so its stored DISTS went stale. A full re-measurement of the current dump at n=627 gives the value printed above; the harness was validated on a cell that is not stale, whose DISTS it reproduces to 6e-6. PSNR, SSIM, FID and LPIPS come from the headline pass, which rescores every cell on every run.
85
+
86
+ ## Load it and translate one SAR PNG
87
+
88
+ ```python
89
+ import numpy as np
90
+ import torch
91
+ from PIL import Image
92
+ from diffusers import UNet2DConditionModel, AutoencoderKL, DDIMScheduler
93
+ from transformers import CLIPTextModel, CLIPTokenizer
94
+
95
+ REPO = "JeonghyeokDo/ReFlowSET" # comparison cells live under baselines/
96
+ BASE = "Manojb/stable-diffusion-2-1-base" # frozen VAE / text encoder / scheduler
97
+ DEV, DT = "cuda", torch.float32
98
+
99
+ unet = UNet2DConditionModel.from_pretrained(
100
+ REPO, subfolder="baselines/sar2opt/sd21ft", torch_dtype=DT).to(DEV).eval()
101
+ vae = AutoencoderKL.from_pretrained(BASE, subfolder="vae", torch_dtype=DT).to(DEV).eval()
102
+ tok = CLIPTokenizer.from_pretrained(BASE, subfolder="tokenizer")
103
+ txt = CLIPTextModel.from_pretrained(BASE, subfolder="text_encoder", torch_dtype=DT).to(DEV).eval()
104
+ sch = DDIMScheduler.from_pretrained(BASE, subfolder="scheduler")
105
+ sch.set_timesteps(50, device=DEV)
106
+
107
+ # The fixed training prompt, tokenized exactly as in training (no padding).
108
+ ids = tok("electro-optical image", padding="do_not_pad",
109
+ max_length=tok.model_max_length, truncation=True,
110
+ return_tensors="pt").input_ids.to(DEV)
111
+ with torch.no_grad():
112
+ embed = txt(ids)[0]
113
+
114
+ # SAR input: 512x512 PNG -> RGB -> [-1, 1]. Do not resize; crop if needed.
115
+ sar = Image.open("sar.png").convert("RGB")
116
+ x = torch.from_numpy(np.array(sar)).permute(2, 0, 1)[None]
117
+ x = (x.to(DEV, DT) / 255.0) * 2 - 1
118
+
119
+ with torch.no_grad():
120
+ sar_lat = vae.encode(x).latent_dist.mean * vae.config.scaling_factor # .mean, not .sample()
121
+ eo_lat = torch.randn_like(sar_lat)
122
+ for t in sch.timesteps:
123
+ out = unet(torch.cat([sar_lat, eo_lat], 1), # SAR first, then the noisy EO latent
124
+ t, encoder_hidden_states=embed).sample
125
+ eo_lat = sch.step(out, t, eo_lat).prev_sample # all 4 channels; no variance head
126
+ eo = vae.decode(eo_lat / vae.config.scaling_factor).sample
127
+ eo = (eo * 0.5 + 0.5).clamp(0, 1)[0].cpu()
128
+
129
+ Image.fromarray((eo.permute(1, 2, 0).numpy() * 255).round().astype(np.uint8)).save("eo.png")
130
+ ```
131
+
132
+ The two things that silently produce a plausible-but-wrong image if you get
133
+ them backwards: the concatenation order is **SAR first**, and the VAE must be
134
+ read as `latent_dist.mean`, not `.sample()`.
135
+
136
+ ## Licence status — stated factually
137
+
138
+ * **Training code:** MIT, `Copyright (c) 2026 KAIST VICLab` (the stage-1
139
+ trainer of [`KAIST-VICLab/C-DiffSET`](https://github.com/KAIST-VICLab/C-DiffSET)).
140
+ The row is named after Stable Diffusion, but no stabilityai code was used and
141
+ no stabilityai repository was loaded.
142
+ * **These weights:** a fine-tune of Stable Diffusion 2.1-base, hence a
143
+ *Derivative of the Model* under the **CreativeML Open RAIL++-M** licence
144
+ (`openrail++`). The **Attachment A use restrictions travel with these
145
+ weights** and must be passed on to anyone you redistribute to. Full text,
146
+ including Attachment A, ships in this repository as `LICENSE-WEIGHTS.md`.
147
+
148
+ Attribution: *fine-tuned from Stable Diffusion 2.1-base (obtained as
149
+ `Manojb/stable-diffusion-2-1-base`, a re-upload of
150
+ `stabilityai/stable-diffusion-2-1-base`) using the stage-1 trainer of
151
+ `KAIST-VICLab/C-DiffSET`.*
152
+
153
+ ---
154
+
155
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
156
+ checkpoint retrained by us on SAR2Opt; it is not ReFlowSET itself. Every
157
+ comparison method in the release was retrained by us on the same splits at the
158
+ same resolution, so the rows are directly comparable to each other.
baselines/sar2opt/sd21ft/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "UNet2DConditionModel",
3
+ "_diffusers_version": "0.10.0.dev0",
4
+ "act_fn": "silu",
5
+ "attention_head_dim": [
6
+ 5,
7
+ 10,
8
+ 20,
9
+ 20
10
+ ],
11
+ "block_out_channels": [
12
+ 320,
13
+ 640,
14
+ 1280,
15
+ 1280
16
+ ],
17
+ "center_input_sample": false,
18
+ "cross_attention_dim": 1024,
19
+ "down_block_types": [
20
+ "CrossAttnDownBlock2D",
21
+ "CrossAttnDownBlock2D",
22
+ "CrossAttnDownBlock2D",
23
+ "DownBlock2D"
24
+ ],
25
+ "downsample_padding": 1,
26
+ "dual_cross_attention": false,
27
+ "flip_sin_to_cos": true,
28
+ "freq_shift": 0,
29
+ "in_channels": 8,
30
+ "layers_per_block": 2,
31
+ "mid_block_scale_factor": 1,
32
+ "norm_eps": 1e-05,
33
+ "norm_num_groups": 32,
34
+ "num_class_embeds": null,
35
+ "only_cross_attention": false,
36
+ "out_channels": 4,
37
+ "sample_size": 64,
38
+ "up_block_types": [
39
+ "UpBlock2D",
40
+ "CrossAttnUpBlock2D",
41
+ "CrossAttnUpBlock2D",
42
+ "CrossAttnUpBlock2D"
43
+ ],
44
+ "use_linear_projection": true
45
+ }
baselines/sar2opt/spade/README.md ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPADE — SAR2Opt
2
+
3
+ **Semantic Image Synthesis with Spatially-Adaptive Normalization**, Park, Liu,
4
+ Wang and Zhu, *CVPR 2019* ([arXiv:1903.07291](https://arxiv.org/abs/1903.07291)).
5
+ Upstream code: [`NVlabs/SPADE`](https://github.com/NVlabs/SPADE) at commit
6
+ `fecacc920c1367a038995c45a39c15f6521ca64f`.
7
+
8
+ This is the spatially-adaptive-normalisation generator we retrained ourselves on
9
+ SAR2Opt, and it is the checkpoint behind the SPADE row of the ReFlowSET
10
+ comparison table.
11
+
12
+ **This checkpoint will not load into a clean clone of `NVlabs/SPADE`.** Read the
13
+ next two sections before trying.
14
+
15
+ ## What is in this folder
16
+
17
+ | file | bytes | what it is |
18
+ |---|---|---|
19
+ | `net_G.pth` | 368,775,707 | the SPADE generator, and the only file inference needs |
20
+
21
+ `net_G.pth` is a **`SPADEGenerator`**: `ngf = 64`,
22
+ `norm_G = spectralspadesyncbatch3x3`, `num_upsampling_layers = normal`,
23
+ `use_vae = False` (no style encoder — the generator is deterministic given the
24
+ input), and **`semantic_nc = 3`**.
25
+
26
+ That `semantic_nc = 3` is the whole story of this cell. SPADE is a *semantic
27
+ image synthesis* model: upstream one-hot-encodes its "label" input into
28
+ `label_nc` channels. A SAR chip is a real image, not a segmentation map, so we
29
+ run with `--label_nc 0 --no_instance`, which upstream treats as an error
30
+ condition. Stock SPADE therefore never constructs a generator with a 3-channel
31
+ spatially-adaptive input and cannot load this file. The two discriminators
32
+ (`num_D` 2) are not released.
33
+
34
+ ## Training budget we used
35
+
36
+ | | |
37
+ |---|---|
38
+ | **generator updates released** | **36,200** |
39
+ | batch size | 8 |
40
+ | epochs (`niter` + `niter_decay`) | 100 + 100 |
41
+ | iterations per epoch | 181 (1,450 training pairs; this loader drops the last partial batch) |
42
+ | optimizer | Adam, lr 2e-4 with TTUR (`no_TTUR` off, so G runs at lr/2 and D at 2·lr), β₁ 0.0, β₂ 0.9 |
43
+ | adversarial loss | hinge GAN, multi-scale discriminator (`num_D` 2, `n_layers_D` 4) |
44
+ | feature-matching loss | λ_feat 10 |
45
+ | perceptual loss | VGG loss, λ_vgg 10 |
46
+ | initialisation | Xavier |
47
+ | augmentation | horizontal flip |
48
+ | input | random 512 crops of the 600 px tiles (`crop`, `load_size` 600, `crop_size` 512) |
49
+
50
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
51
+
52
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
53
+ |---|---|---|---|---|
54
+ | 142.5 | 0.265 | 0.597 | 0.234 | 14.47 |
55
+
56
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
57
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
58
+ method's training and in the evaluation. No subsampling: every metric on this
59
+ page is measured over all 627 pairs.
60
+
61
+ These are our own re-evaluation numbers, measured by us on the images this
62
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
63
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
64
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
65
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
66
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
67
+ convention gives a systematically different number and the two must never be
68
+ mixed, or compared against a paper that used the other one.
69
+
70
+ ## Load it and translate one SAR image
71
+
72
+ **You need the image-to-image adaptation described below**, applied to the
73
+ upstream repository, before this will run. With that in place, the generator can
74
+ be constructed directly:
75
+
76
+ ```python
77
+ import numpy as np, torch
78
+ from argparse import Namespace
79
+ from PIL import Image
80
+ from models.networks.generator import SPADEGenerator # from the NVlabs/SPADE repo
81
+
82
+ opt = Namespace(semantic_nc=3, label_nc=0, no_instance=True, ngf=64,
83
+ num_upsampling_layers='normal',
84
+ norm_G='spectralspadesyncbatch3x3',
85
+ crop_size=512, aspect_ratio=1.0, use_vae=False)
86
+ G = SPADEGenerator(opt)
87
+ G.load_state_dict(torch.load('net_G.pth', map_location='cpu'))
88
+ G.eval()
89
+
90
+ sar = Image.open('sar.png').convert('RGB')
91
+ sar = sar.crop(((600-512)//2, (600-512)//2, (600+512)//2, (600+512)//2))
92
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
93
+
94
+ with torch.no_grad():
95
+ y = G(seg=x, z=None) # the SAR image goes in the 'seg' slot
96
+
97
+ eo = ((y[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
98
+ Image.fromarray(eo).save('eo.png')
99
+ ```
100
+
101
+ Through the repository's own `test.py`, the invocation is
102
+ `--dataset_mode custom --label_nc 0 --no_instance --no_pairing_check` with the
103
+ SAR directory as `--label_dir`. Note that the paired loader also demands an
104
+ `--image_dir` even at test time, so a SAR-only run needs either a dummy EO
105
+ directory or the direct generator call above.
106
+
107
+ ## Read before using this checkpoint
108
+
109
+ **The five changes that make this cell exist.** These are a functional
110
+ adaptation, not compatibility fixes, and a reproducer must apply all of them:
111
+
112
+ 1. `data/pix2pix_dataset.py` — when `label_nc == 0`, load the "label" through
113
+ the *image* transform (`.convert('RGB')`, normalised to `[-1, 1]`) instead of
114
+ nearest-neighbour resampling, ×255 and the `==255 → label_nc` remap. The SAR
115
+ PNG is a real image.
116
+ 2. `models/pix2pix_model.py` — an early return in `preprocess_input()`: when
117
+ `label_nc == 0`, skip the `.long()` cast and the `scatter_` one-hot encoding
118
+ and pass the float image straight through as `input_semantics`. One-hot
119
+ encoding a normalised float image is undefined.
120
+ 3. `options/base_options.py` — when `label_nc == 0`, force
121
+ `opt.semantic_nc = 3 + (0 if no_instance else 1)`. Upstream computes
122
+ `semantic_nc = label_nc + …`, i.e. 0 or 1, and would build a generator with a
123
+ one-channel spatially-adaptive input.
124
+ 4. `models/networks/discriminator.py` — the discriminator input width becomes
125
+ `(label_nc if label_nc > 0 else 3) + output_nc`, for the same reason on the
126
+ discriminator side.
127
+ 5. `util/visualizer.py` — `tensor2label` is given `n_label=0` when
128
+ `label_nc == 0` so it falls through to `tensor2im` instead of colour-mapping
129
+ a real image as a label map; and `import scipy.misc` is guarded, since it was
130
+ removed in SciPy ≥ 1.12.
131
+
132
+ Changes 1–4 change what the network *is*. **Without them this file does not
133
+ load.**
134
+
135
+ * Normalisation is `[-1, 1]`.
136
+ * `norm_G` uses synchronised batch normalisation; on a single device this
137
+ behaves as ordinary batch norm, and `.eval()` is required.
138
+ * Passes the identity-collapse audit on both datasets.
139
+
140
+ ## Licence — stated factually ⚠ non-commercial and share-alike
141
+
142
+ Upstream `NVlabs/SPADE` ships `LICENSE.md`, reproduced verbatim here as
143
+ `licenses/LICENSE-spade.txt`. It is the full legal code of
144
+ **Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International
145
+ (CC BY-NC-SA 4.0)**. The copyright holder is named in the README rather than in
146
+ the licence file: *"Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
147
+ … The code is released for academic research use only. For commercial use or
148
+ business inquiries, please contact researchinquiries@nvidia.com."*
149
+
150
+ **This is the most restrictive licence in this repository, and two of its terms
151
+ travel with this checkpoint:**
152
+
153
+ * **NonCommercial.** You may not use this checkpoint for commercial advantage or
154
+ monetary compensation.
155
+ * **ShareAlike.** If a trained checkpoint is Adapted Material — and we treat it
156
+ as such — then any licence you apply when you pass it on must be a Creative
157
+ Commons licence with the same elements, this version or later, or a BY-NC-SA
158
+ compatible licence. **This cell is therefore distributed under CC BY-NC-SA
159
+ 4.0**; it cannot sit under a permissive repository-wide weight licence, and
160
+ neither can anything you derive from it.
161
+ * The licence also states that patent and trademark rights are not licensed.
162
+ * NVIDIA's academic-use-only statement in the README is an additional assertion
163
+ beyond the CC text.
164
+
165
+ Attribution: *trained with `NVlabs/SPADE`. Copyright (C) 2019 NVIDIA
166
+ Corporation, all rights reserved. Licensed under CC BY-NC-SA 4.0; NVIDIA
167
+ releases the code for academic research use only (commercial enquiries:
168
+ researchinquiries@nvidia.com). This checkpoint is therefore distributed under
169
+ CC BY-NC-SA 4.0 — non-commercial, share-alike. Full text in
170
+ `licenses/LICENSE-spade.txt`.*
171
+
172
+ ---
173
+
174
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
175
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
176
+ itself. Every comparison method in this repository was retrained by us on the
177
+ same splits at the same resolution and scored through one evaluation pipeline,
178
+ so the rows are directly comparable to each other — and, for the same reason,
179
+ **not** directly comparable with the numbers in the methods' own papers.
baselines/sar2opt/stegogan/README.md ADDED
@@ -0,0 +1,205 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # StegoGAN — SAR2Opt
2
+
3
+ **StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image
4
+ Translation**, Wu, Chen, Mermet, Hurni, Schindler, Gonthier and Landrieu,
5
+ *CVPR 2024*. Upstream code:
6
+ [`sian-wusidi/StegoGAN`](https://github.com/sian-wusidi/StegoGAN) (the official
7
+ CVPR 2024 repository) at commit `cad61997c0f82793444f60f81298142b80cdf3c1`.
8
+
9
+ This is the mismatch-aware unpaired GAN we retrained ourselves on SAR2Opt,
10
+ and it is the checkpoint behind the StegoGAN row of the ReFlowSET comparison
11
+ table. **Two things on this page are load-bearing: there is an oracle output you
12
+ must not use, and this cell fails our input-copy audit.**
13
+
14
+ We applied **no patches** to this repository. It is stock upstream at that
15
+ commit.
16
+
17
+ ## What is in this folder
18
+
19
+ | file | bytes | what it is |
20
+ |---|---|---|
21
+ | `net_G_A.pth` | 50,254,749 | **SAR → EO** (`resnet_9blocks_maskv1`) — the generator that produces the reported row |
22
+ | `net_G_B.pth` | 52,615,525 | EO → SAR (`resnet_9blocks_maskv3`), which also returns the mismatch mask; used to close the cycle in training, and **not needed for a prediction** |
23
+
24
+ Both generators are ResNet-style with instance normalisation, `ngf = 64`, built
25
+ with `--resnet_layer 8 --fusionblock`. **Those two are architecture flags, not
26
+ training options**: build the network without them and the state dict will not
27
+ load.
28
+
29
+ `net_G_A` takes an **optional second argument**. That option is the whole reason
30
+ this card has a warning section — see below.
31
+
32
+ ## Training budget we used
33
+
34
+ | | |
35
+ |---|---|
36
+ | **data iterations released** | **87,000** |
37
+ | generator optimizer steps | 174,000 — see the note below |
38
+ | batch size | 2 |
39
+ | epochs (constant LR + linear decay) | 60 + 60 |
40
+ | iterations per epoch | 725 (1,450 training images, last batch not dropped) |
41
+ | optimizer | Adam, lr 2e-4, β₁ 0.5, linear decay to 0 over the second half |
42
+ | adversarial loss | LSGAN; discriminator `basic` (70×70 PatchGAN) |
43
+ | cycle losses | λ_A 10, λ_B 10 |
44
+ | identity loss | λ_identity 0.5 |
45
+ | consistency / regularisation | λ_consistency 1.0, λ_reg 0.3 |
46
+ | augmentation | horizontal flip |
47
+ | input | random 512 crops of the 600 px tiles |
48
+
49
+ **Why two numbers.** StegoGAN's training step calls `optimizer_G.step()`
50
+ **twice** off a single backward pass, so its optimizer-step count is exactly
51
+ double its data-iteration count. We quote StegoGAN budgets in *data iterations*
52
+ throughout this benchmark and note the 2× explicitly. Do not re-derive one cell
53
+ the other way — the family stops being comparable with itself.
54
+
55
+ ## Measured on the SAR2Opt test set (n = 627, 512 px)
56
+
57
+ | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
58
+ |---|---|---|---|---|
59
+ | 150.1 | 0.347 | 0.655 | 0.158 | 12.47 |
60
+
61
+ Evaluated on the official split's 627 test tiles, centre-cropped to 512 px.
62
+ The tiles are 600 px natively; this benchmark crops and never resizes, in every
63
+ method's training and in the evaluation. No subsampling: every metric on this
64
+ page is measured over all 627 pairs.
65
+
66
+ These are our own re-evaluation numbers, measured by us on the images this
67
+ checkpoint produced. **No number here is copied from any paper.** PSNR and SSIM
68
+ are per-image torchmetrics with `data_range=1`; LPIPS is **LPIPS-VGG on inputs
69
+ mapped to [-1, 1]** (the `normalize=False` convention); DISTS is the standard
70
+ implementation on the same pairs; FID is `pytorch-fid` against the size-matched
71
+ ground-truth test set. The LPIPS convention matters: the alternative [0, 1]
72
+ convention gives a systematically different number and the two must never be
73
+ mixed, or compared against a paper that used the other one.
74
+
75
+ ## Load it and translate one SAR image
76
+
77
+ Clone `sian-wusidi/StegoGAN` at the commit above and run this from its root.
78
+ Check `define_G`'s signature in that repository first: StegoGAN adds
79
+ `resnet_layer` and `fusionblock` keyword arguments that the junyanz original does
80
+ not have.
81
+
82
+ ```python
83
+ import numpy as np, torch
84
+ from PIL import Image
85
+ from models.networks import define_G # from the StegoGAN repo
86
+
87
+ G_A = define_G(3, 3, 64, 'resnet_9blocks_maskv1', 'instance', use_dropout=False,
88
+ init_type='normal', init_gain=0.02, gpu_ids=[],
89
+ resnet_layer=8, fusionblock=True)
90
+ G_A.load_state_dict(torch.load('net_G_A.pth', map_location='cpu'))
91
+ G_A.eval()
92
+
93
+ sar = Image.open('sar.png').convert('RGB')
94
+ sar = sar.crop(((600-512)//2, (600-512)//2, (600+512)//2, (600+512)//2))
95
+ x = torch.from_numpy(np.asarray(sar, np.float32) / 127.5 - 1).permute(2, 0, 1)[None]
96
+
97
+ with torch.no_grad():
98
+ fake_B_clean = G_A(x) # ONE argument. Passing a second makes it an oracle.
99
+
100
+ eo = ((fake_B_clean[0].permute(1, 2, 0).numpy() + 1) * 127.5).clip(0, 255).astype(np.uint8)
101
+ Image.fromarray(eo).save('eo.png')
102
+ ```
103
+
104
+ Through the repository's own `test.py` (`--model stego_gan --no_dropout
105
+ --resnet_layer 8 --fusionblock`), outputs are written one directory per visual;
106
+ **`fake_B_clean/` is the row on this page.**
107
+
108
+ ## ⚠ `fake_B` is an oracle. `fake_B_clean` is the prediction.
109
+
110
+ The model computes two outputs:
111
+
112
+ ```
113
+ fake_B_clean = netG_A(real_A) # SAR only
114
+ fake_B = netG_A(real_A, latent_real_B.detach()) # + a feature map of the GROUND TRUTH
115
+ ```
116
+
117
+ `latent_real_B` comes from `netG_B(real_B)` — the *target* EO image. `fake_B` is
118
+ therefore conditioned on the answer and **cannot be produced at deployment**.
119
+ Only `fake_B_clean` is a prediction, and every number on this page is
120
+ `fake_B_clean`.
121
+
122
+ This is not hypothetical. Earlier in our own benchmarking the oracle was scored as
123
+ "StegoGAN" for two days on a different dataset and topped every column at
124
+ 23.19 dB, against a true deployable value of 12.74 dB. **Any published StegoGAN
125
+ number must state which of the two it is.**
126
+
127
+ ## ⚠ This cell fails the input-copy audit
128
+
129
+ We audit every cell for **identity collapse**: a model that has learned to pass
130
+ its input through rather than translate it. The test compares the mean absolute
131
+ distance from the generated image to the SAR **input** against the distance to
132
+ the EO **target**; a ratio below 1.0 means the output is closer to the input it
133
+ was given than to the answer it was asked for.
134
+
135
+ | quantity | value |
136
+ |---|---|
137
+ | mean \|generated − ground truth\| | 48.694 |
138
+ | mean \|generated − SAR input\| | 36.915 |
139
+ | **ratio** (SAR distance ÷ GT distance) | **0.758** — below 1.0 |
140
+ | verdict | **COLLAPSED** |
141
+ | cross-pair separation | 0.890 (cohort median 1.088) — no oracle behaviour |
142
+
143
+ **Read the metrics on this page with that in mind.** They are reported unchanged
144
+ because they are what the released implementation does at its own published
145
+ protocol, and a benchmark that quietly substitutes a better-behaved variant is
146
+ not reporting the method. But they should not be read as translation quality.
147
+ The separation figure is the companion check for the *opposite* failure: it is
148
+ the mean distance from a generated image to **other** images' ground truth
149
+ divided by the distance to its **own**, so a value well above the cohort median
150
+ would indicate that ground-truth information had leaked into the generation.
151
+ This cell sits at or below the median, so there is no leak — the problem here is
152
+ input-copying, and only input-copying.
153
+
154
+ A longer control run at roughly 1.85× this budget was also trained as a diagnostic; it collapses harder (ratio 0.697), and it is neither the table's row nor published here.
155
+
156
+ ## Read before using this checkpoint
157
+
158
+ * **`--resnet_layer 8` and `--fusionblock` must match training.** They change the
159
+ architecture; omit them and the load fails.
160
+ * **`test.py` rejects `--display_id`.** That flag is declared only in the train
161
+ options. Passing it to `test.py` aborts with an unrecognised argument — and in
162
+ a shell `&&` chain the cell then reports success with an empty output
163
+ directory. This silently destroyed several of our own test passes once.
164
+ * Build with `gpu_ids=[]`; the checkpoints have bare keys.
165
+ * Normalisation is `[-1, 1]`.
166
+ * StegoGAN is **unpaired**.
167
+
168
+ ## Licence — stated factually ⚠ no upstream licence exists
169
+
170
+ **The upstream code base publishes no licence.**
171
+ [`sian-wusidi/StegoGAN`](https://github.com/sian-wusidi/StegoGAN) — the official
172
+ CVPR 2024 repository — has no LICENSE, LICENCE, COPYING or NOTICE file at any
173
+ depth; **0 of its 52 tracked files** at the pinned commit are licence-shaped; its
174
+ README has no licence section; and the GitHub API reports no declared licence,
175
+ with the `/license` endpoint returning 404. Checked 2026-08-28.
176
+
177
+ Under default copyright that means **all rights are reserved by the authors and
178
+ no express permission to redistribute derived work has been granted** to us or
179
+ to you. We publish this checkpoint anyway, so that the benchmark is reproducible
180
+ end to end, and we state the position plainly rather than implying a permission
181
+ that does not exist. If you intend to redistribute this checkpoint or build on
182
+ it, assess that for yourself, and consider asking the authors directly.
183
+
184
+ **What is known about the lineage.** StegoGAN's README thanks
185
+ `junyanz/pytorch-CycleGAN-and-pix2pix`, and the parts of the tree that are
186
+ unmodified CycleGAN carry that project's BSD notice, reproduced here as
187
+ `licenses/LICENSE-cyclegan.txt`. **That notice does not extend to StegoGAN's own
188
+ contributions** — the mismatch-mask generators, the fusion block and the
189
+ consistency and regularisation losses, which are exactly what makes
190
+ `net_G_A.pth` what it is.
191
+
192
+ Please cite: Wu, Chen, Mermet, Hurni, Schindler, Gonthier and Landrieu,
193
+ *StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image
194
+ Translation*, CVPR 2024.
195
+ The full record of what we checked, per method, is in
196
+ `licenses/NO-UPSTREAM-LICENSE.md` at the root of this repository.
197
+
198
+ ---
199
+
200
+ Part of the **ReFlowSET** release. This folder holds one comparison-method
201
+ checkpoint that we retrained ourselves on SAR2Opt; it is not ReFlowSET
202
+ itself. Every comparison method in this repository was retrained by us on the
203
+ same splits at the same resolution and scored through one evaluation pipeline,
204
+ so the rows are directly comparable to each other — and, for the same reason,
205
+ **not** directly comparable with the numbers in the methods' own papers.