---
license: other
license_name: per-method-see-below
library_name: pytorch
pipeline_tag: image-to-image
tags:
  - sar-to-eo
  - image-translation
  - remote-sensing
  - baselines
  - benchmark
  - reflowset
---

# ReFlowSET — retrained comparison-method checkpoints

The fifteen comparison methods of the **ReFlowSET** SAR-to-EO benchmark, on both
of its datasets: **QXS-SAROPT** (256 px) and **SAR2Opt** (512 px). Thirty
checkpoint cells in total.

**Every checkpoint here was retrained by us.** None is an original author's
released checkpoint. Each method was trained on the same official split, at the
same resolution, under a budget stated in optimizer updates, and every cell was
then scored through one evaluation pipeline with an identity-collapse and
ground-truth-leak audit. That is what makes the rows below comparable **to each
other** — and, for exactly the same reason, not comparable with the numbers in
the methods' own papers.

ReFlowSET itself is not in this repository. It lives at
[JeonghyeokDo/ReFlowSET](https://huggingface.co/JeonghyeokDo/ReFlowSET); code at https://github.com/KAIST-VICLab/ReFlowSET; project page
at https://kaist-viclab.github.io/ReFlowSET_site/.

## Results — all sixteen rows, both datasets

Our own measurements, on the images each checkpoint produced. No number below
is copied from any paper.

### QXS-SAROPT — n = 3,999, 256 px

| Method | Venue | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
|---|---|---|---|---|---|---|
| *General image-to-image* | | | | | | |
| pix2pix | CVPR'17 | 174.6 | 0.373 | 0.665 | 0.203 | 12.33 |
| CycleGAN ⚠ | ICCV'17 | 104.4 | 0.376 | 0.653 | 0.262 | 12.92 |
| pix2pixHD | CVPR'18 | 85.7 | 0.298 | 0.573 | 0.358 | 16.13 |
| SPADE | CVPR'19 | 90.7 | 0.292 | 0.599 | 0.320 | 14.53 |
| DDPM (SR3-class) | TPAMI'22 | 43.8 | 0.311 | 0.620 | 0.359 | 14.04 |
| SD2.1 fine-tune only | CVPR'22 | 19.1 | 0.257 | 0.561 | 0.348 | 15.40 |
| BBDM | CVPR'23 | 76.6 | 0.270 | 0.568 | 0.352 | 15.34 |
| ControlNet | ICCV'23 | 50.4 | 0.307 | 0.604 | 0.297 | 13.42 |
| HI-Diff | NeurIPS'23 | 324.3 | 0.539 | 0.692 | 0.457 | 17.10 |
| ResShift | NeurIPS'23 | 140.2 | 0.334 | 0.607 | 0.217 | 14.20 |
| StegoGAN ⚠ | CVPR'24 | 106.8 | 0.384 | 0.658 | 0.254 | 12.96 |
| *SAR-to-EO* | | | | | | |
| Conditional Diffusion | GRSL'24 | 88.6 | 0.355 | 0.730 | 0.213 | 11.55 |
| cBBDM | GRSL'25 | 50.6 | 0.246 | 0.539 | 0.372 | 16.02 |
| E3Diff | GRSL'25 | 47.8 | 0.278 | 0.530 | 0.302 | 16.44 |
| C-DiffSET | TCSVT'26 | 19.9 | 0.233 | 0.526 | 0.380 | 16.92 |
| *Ours* | | | | | | |
| **ReFlowSET (ours)** | — | 19.1 | 0.231 | 0.534 | 0.355 | 16.09 |

### SAR2Opt — n = 627, 512 px (centre crops of the 600 px tiles)

| Method | Venue | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
|---|---|---|---|---|---|---|
| *General image-to-image* | | | | | | |
| pix2pix | CVPR'17 | 261.9 | 0.347 | 0.657 | 0.199 | 13.39 |
| CycleGAN ⚠ | ICCV'17 | 143.5 | 0.330 | 0.650 | 0.178 | 12.90 |
| pix2pixHD | CVPR'18 | 146.3 | 0.283 | 0.567 | 0.268 | 15.95 |
| SPADE | CVPR'19 | 142.5 | 0.265 | 0.597 | 0.234 | 14.47 |
| DDPM (SR3-class) | TPAMI'22 | 122.5 | 0.295 | 0.610 | 0.313 | 13.65 |
| SD2.1 fine-tune only | CVPR'22 | 71.8 | 0.211 † | 0.541 | 0.293 | 16.24 |
| BBDM | CVPR'23 | 143.1 | 0.290 | 0.590 | 0.276 | 15.29 |
| ControlNet | ICCV'23 | 140.5 | 0.350 | 0.643 | 0.217 | 11.73 |
| HI-Diff | NeurIPS'23 | 319.8 | 0.473 | 0.692 | 0.384 | 17.36 |
| ResShift | NeurIPS'23 | 141.7 | 0.304 | 0.597 | 0.177 | 14.31 |
| StegoGAN ⚠ | CVPR'24 | 150.1 | 0.347 | 0.655 | 0.158 | 12.47 |
| *SAR-to-EO* | | | | | | |
| Conditional Diffusion | GRSL'24 | 211.8 | 0.415 | 0.686 | 0.248 | 12.48 |
| cBBDM | GRSL'25 | 222.3 | 0.377 | 0.571 | 0.361 | 17.05 |
| E3Diff | GRSL'25 | 104.7 | 0.232 | 0.529 | 0.249 | 16.09 |
| C-DiffSET | TCSVT'26 | 78.1 | 0.214 † | 0.529 | 0.314 | 16.81 |
| *Ours* | | | | | | |
| **ReFlowSET (ours)** | — | 66.3 | 0.185 | 0.522 | 0.287 | 16.06 |

⚠ **fails the input-copy audit** — see *Identity collapse* below.
† **these two DISTS values were re-measured.** The image dumps for these two
SAR2Opt cells were regenerated after their last extended-metrics pass, and that
pass's resume check keys on the number of images rather than on their content, so
it skipped them and their stored DISTS went stale. Both were re-measured on the
current dumps at full n=627 and the table prints the re-measurement (SD2.1-FT
0.212 -> **0.211**; C-DiffSET 0.2141 -> **0.2136**, unchanged at three decimals).
The harness was validated on a cell that is *not* stale, whose DISTS it
reproduces to 6e-6. FID, LPIPS, SSIM and PSNR come from a pass that rescores
every cell on every run and were never affected.

## Read before you compare these numbers with a paper

**These numbers are not comparable with the source papers, and are not meant to
be.** Three reasons, all deliberate:

1. **Budgets are matched across the benchmark, not to each method's own
   recipe.** Several methods are trained here at a fraction of their published
   schedule — HI-Diff, for instance, is trained for 50,000 updates against an
   upstream recipe of 600,000, with progressive resizing disabled. Where we
   departed from a published recipe, the method's own card says so and says by
   how much.
2. **Budgets are quoted in optimizer updates, never epochs.** The two datasets
   differ by more than tenfold in size (16,001 versus 1,450 training pairs), so
   an epoch count means something completely different on each. Comparing
   unpaired methods by epochs produced two successive wrong conclusions in our
   own work before we changed the unit.
3. **One evaluation protocol for everyone.** Same test items, same resolution,
   same metric implementations, same ground truth. SAR2Opt is **centre-cropped**
   to 512 px and never resized, in training and in evaluation alike.

### The LPIPS convention

LPIPS here is **LPIPS-VGG on inputs mapped to `[-1, 1]`** before the network —
`lpips_vgg(x * 2 - 1, y * 2 - 1)` with the library's default `normalize=False`.
The other convention, which several released evaluators use, hands the network
`[0, 1]` inputs *unscaled* (still `normalize=False`) and produces a
**systematically lower number** — about 0.05 lower — on the same images. Note
that `[0, 1]` inputs with `normalize=True` is the *same* thing as ours, not the
other one; only the flag differs. Both are called "LPIPS" in the literature. If you compare a number from
this table against one from a paper, check which convention that paper used
first; otherwise the comparison is meaningless. The same care applies in reverse
to anyone quoting these numbers.

PSNR and SSIM are per-image torchmetrics with `data_range=1`. FID is
`pytorch-fid` against the size-matched ground-truth test set. DISTS is the
standard implementation on the same pairs. Where the generated size and the
ground-truth size differ, the ground truth is centre-cropped to match.

### Identity collapse

**Four of the thirty cells fail an input-copy audit**: CycleGAN and StegoGAN, on
both datasets. Their outputs are closer to the SAR image they were given than to
the EO image they were asked for — mean |output − SAR| < mean |output − ground
truth| — which is the signature of a model that has learned to pass its input
through rather than translate it.

| cell | ratio (SAR distance ÷ GT distance) | verdict |
|---|---|---|
| QXS-SAROPT CycleGAN | 0.847 | COLLAPSED |
| QXS-SAROPT StegoGAN | 0.919 | COLLAPSED |
| SAR2Opt CycleGAN | 0.795 | COLLAPSED |
| SAR2Opt StegoGAN | 0.758 | COLLAPSED |

**Their metric values are reported unchanged and marked in place.** They are what
the released implementation does at its own published protocol, and a benchmark
that quietly substitutes a better-behaved variant is not reporting the method.
But they should not be read as translation quality. No other cell on either
dataset collapses — in particular DDPM, Conditional Diffusion and BBDM do **not**
collapse here, whatever they may do elsewhere.

The companion leak audit is clean: no cell on either dataset shows a generation
that matches its own ground truth anomalously better than it matches other
images' ground truth. Per-cell numbers are on each method's card.

**One oracle you must not use.** StegoGAN's `fake_B` output is computed as
`netG_A(real_A, latent(real_B))` — conditioned on the ground truth, and
impossible to produce at deployment. Only `fake_B_clean = netG_A(real_A)` is a
prediction, and it is what the StegoGAN rows above report. Any published StegoGAN
number should say which of the two it is.

## Layout

```
<dataset>/<method>/<weight files> + README.md
dataset ∈ { qxs-saropt, sar2opt }
licenses/          third-party licence texts, one per method
```

Every one of the thirty folders carries its own `README.md` with that cell's
architecture, the file-by-file contents, the training budget in optimizer
updates, the measured metrics, a copy-pasteable load-and-translate snippet, the
traps specific to that method, and its licence stated factually.

| method | folder | files | upstream code |
|---|---|---|---|
| pix2pix | `pix2pix/` | `net_G.pth` | `junyanz/pytorch-CycleGAN-and-pix2pix` |
| CycleGAN | `cyclegan/` | `net_G_A.pth` (**SAR→EO**), `net_G_B.pth` | `junyanz/pytorch-CycleGAN-and-pix2pix` |
| pix2pixHD | `p2phd/` | `net_G.pth` | `NVIDIA/pix2pixHD` |
| SPADE | `spade/` | `net_G.pth` | `NVlabs/SPADE` |
| DDPM (SR3-class) | `ddpm/` | `gen.pth` | `DeepSARRS/E3Diff`, stage 1 |
| SD2.1 fine-tune only | `sd21ft/` | `diffusion_pytorch_model.safetensors`, `config.json` | `KAIST-VICLab/C-DiffSET`, stage 1, on SD 2.1-base |
| BBDM | `bbdm/` | `last_model.pth` | `xuekt98/BBDM` |
| ControlNet | `controlnet/` | `diffusion_pytorch_model.safetensors`, `config.json` | `huggingface/diffusers` `train_controlnet.py`, on SD 2.1-base |
| HI-Diff | `hidiff/` | `S1_*.pth` ×2, `S2_*.pth` ×3 (**only the three `S2_` files are needed at inference**) | `zhengchen1999/HI-Diff` |
| ResShift | `resshift/` | `ema_model.pth` | `zsyOAOA/ResShift` |
| StegoGAN | `stegogan/` | `net_G_A.pth` (**SAR→EO**), `net_G_B.pth` | `sian-wusidi/StegoGAN` |
| Conditional Diffusion | `conddiff/` | `ema_final.pt` | `Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation` |
| cBBDM | `cbbdm/` | `last_model.pth` | `egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical` |
| E3Diff | `e3diff/` | `gen.pth` | `DeepSARRS/E3Diff`, stage 2 |
| C-DiffSET | `cdiffset/` | `diffusion_pytorch_model.safetensors`, `config.json` | `KAIST-VICLab/C-DiffSET`, stage 2 |

### Three checkpoints these weights need and this repository does not carry

* **BBDM and cBBDM** need the CompVis latent-diffusion **vq-f4 VQGAN**
  (`model.ckpt`, 756,175,527 bytes) at the path named in their config — at
  construction time, even though the same weights are also inside
  `last_model.pth`. MIT.
* **ResShift** needs the CompVis **vq-f4 autoencoder** (221,364,711 bytes). MIT.
* **SD2.1-FT, ControlNet and C-DiffSET** need Stable Diffusion 2.1-base for the
  frozen VAE, text encoder, tokenizer and scheduler. Their cards name the exact
  repository we loaded.
* **DDPM and E3Diff** need no extra checkpoint, but do need **PPB-despeckled and
  Canny condition images** computed from each SAR input; neither can be run from
  a SAR file alone.

## Licences

**A single repository-wide licence tag would be wrong for this collection.** The
fifteen methods fall into three tiers, and the restrictive terms in tiers B and C
travel with the individual cells. Each method's card states its position in full;
the licence texts are in `licenses/`.

| method | upstream licence | redistribution | what propagates |
|---|---|---|---|
| pix2pix | multi-part BSD (CycleGAN / pix2pix / DCGAN notices) | ✅ with the notice | no-endorsement (DCGAN part) |
| CycleGAN | same file as pix2pix | ✅ with the notice | same |
| pix2pixHD | BSD (NVIDIA 2019) + bundled CycleGAN notice | ✅ with both notices | none |
| **SPADE** | **CC BY-NC-SA 4.0** | ⚠️ **non-commercial only** | **NonCommercial + ShareAlike** — this cell is itself distributed under CC BY-NC-SA 4.0 |
| **DDPM (SR3-class)** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown; there is no licence to read |
| SD2.1 fine-tune only | MIT code (C-DiffSET) / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |
| BBDM | MIT | ✅ | none (needs the external vq-f4 VQGAN, MIT) |
| ControlNet | Apache-2.0 code (diffusers) / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |
| HI-Diff | Apache-2.0 | ✅ | ship the licence, keep notices, state your modifications |
| **ResShift** | **S-Lab License 1.0** | ⚠️ **non-commercial only** | **NonCommercial**; no-endorsement |
| **StegoGAN** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown |
| **Conditional Diffusion** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown (the guided-diffusion parts it forks are MIT) |
| cBBDM | MIT | ✅ | none (needs the external vq-f4 VQGAN, MIT) |
| **E3Diff** | **none declared** | ⚠️ **no express grant** — published by our decision | unknown |
| C-DiffSET | MIT code / **Open RAIL++-M** weights | ✅ under Open RAIL++-M | **Attachment A use restrictions** |

### The four with no upstream licence

**StegoGAN, E3Diff, DDPM (SR3-class) and Conditional Diffusion are derived from
upstream code that publishes no licence at all** — no LICENSE, LICENCE, COPYING
or NOTICE file, no licence section in the README, and no licence declared to
GitHub. Under default copyright, all rights remain reserved by those authors and
**no express permission to redistribute derived work has been granted** to us or
to you.

We publish these four anyway, so that the benchmark is reproducible end to end,
and we say so plainly rather than implying a permission that does not exist. If
you intend to redistribute them or build on them, assess that for yourself, and
consider asking the upstream authors directly.
`licenses/NO-UPSTREAM-LICENSE.md` records, per method, exactly what we checked
and exactly what we found.

### Files in `licenses/`

`LICENSE-pix2pix.txt`, `LICENSE-cyclegan.txt`, `LICENSE-p2phd.txt`,
`LICENSE-spade.txt`, `LICENSE-bbdm.txt`, `LICENSE-cbbdm.txt`,
`LICENSE-hidiff.txt`, `LICENSE-resshift.txt`, `LICENSE-cdiffset.txt`,
`LICENSE-sd21ft.txt` — one per method, copied verbatim from the upstream
repository we trained with. `LICENSE-SoftPool-MIT.txt` is a vendored dependency
of the E3Diff code base and is included because it is genuinely part of what
those two cells were trained with; it is **not** a licence for E3Diff.
`NO-UPSTREAM-LICENSE.md` covers the four methods with no grant.
`LICENSE-WEIGHTS.md` at the repository root is the CreativeML Open RAIL++-M
text, including Attachment A, which governs the three Stable-Diffusion-derived
cells.

## Datasets

Neither corpus is redistributed here, in any form, and neither may be
redistributed from here.

* **QXS-SAROPT** publishes no licence file; its one stated term is a citation
  requirement — *"This paper must be cited when the dataset is used for research
  purposes"* (Huang et al., [arXiv:2103.08259](https://arxiv.org/abs/2103.08259)).
  Its distribution is request-gated by the authors. Obtain it from them.
* **SAR2Opt** ships an MIT LICENSE, `Copyright (c) 2021 MarsZhaoYT`. The MIT
  text grants rights in "the Software"; the repository does not name the imagery
  sources, so that file is best read as covering the authors' packaging and code.
  Citation requested: Zhao, Celik, Liu and Li, *A Comparative Analysis of
  GAN-based Methods for SAR-to-Optical Image Translation*, IEEE GRSL, 2022.
* **The optical side of both datasets is Google-Earth-derived.** Copyright in
  that imagery is held by third parties, not by the dataset authors and not by
  us. No such imagery appears in this repository.

## A note on two venue labels

The table above prints the venue strings as they appear in the ReFlowSET paper.
Two of them do not match the year in the upstream repositories' own citation
blocks, and both refer to the same paper under an early-access versus issue year:

* **Conditional Diffusion** — the authors' bibtex says **2023**,
  [doi:10.1109/LGRS.2023.3337143](https://doi.org/10.1109/LGRS.2023.3337143).
* **E3Diff** — the authors' bibtex says **2024**,
  [doi:10.1109/LGRS.2024.3506566](https://doi.org/10.1109/LGRS.2024.3506566).

Cite by DOI.

## Citation

If you use this benchmark or any of these weights, please cite ReFlowSET:

```bibtex
@article{do2026reflowset,
  title   = {ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation},
  author  = {Jeonghyeok Do<sup>1</sup>, Seungchul Lee<sup>2</sup>, Munchurl Kim<sup>1*</sup>},
  journal = {arXiv preprint arXiv:2609.00968},
  year    = {2026}
}
```

**And please cite each method's original paper** — every model card carries the
reference — **and both dataset papers.** The QXS-SAROPT citation is required by
that dataset's own terms.