baselines/README.md

metadata

license: other
license_name: per-method-see-below
library_name: pytorch
pipeline_tag: image-to-image
tags:
  - sar-to-eo
  - image-translation
  - remote-sensing
  - baselines
  - benchmark
  - reflowset

ReFlowSET — retrained comparison-method checkpoints

The fifteen comparison methods of the ReFlowSET SAR-to-EO benchmark, on both of its datasets: QXS-SAROPT (256 px) and SAR2Opt (512 px). Thirty checkpoint cells in total.

Every checkpoint here was retrained by us. None is an original author's released checkpoint. Each method was trained on the same official split, at the same resolution, under a budget stated in optimizer updates, and every cell was then scored through one evaluation pipeline with an identity-collapse and ground-truth-leak audit. That is what makes the rows below comparable to each other — and, for exactly the same reason, not comparable with the numbers in the methods' own papers.

ReFlowSET itself is not in this repository. It lives at JeonghyeokDo/ReFlowSET; code at https://github.com/KAIST-VICLab/ReFlowSET; project page at https://kaist-viclab.github.io/ReFlowSET_site/.

Results — all sixteen rows, both datasets

Our own measurements, on the images each checkpoint produced. No number below is copied from any paper.

QXS-SAROPT — n = 3,999, 256 px

Method	Venue	FID↓	DISTS↓	LPIPS↓	SSIM↑	PSNR↑
General image-to-image
pix2pix	CVPR'17	174.6	0.373	0.665	0.203	12.33
CycleGAN ⚠	ICCV'17	104.4	0.376	0.653	0.262	12.92
pix2pixHD	CVPR'18	85.7	0.298	0.573	0.358	16.13
SPADE	CVPR'19	90.7	0.292	0.599	0.320	14.53
DDPM (SR3-class)	TPAMI'22	43.8	0.311	0.620	0.359	14.04
SD2.1 fine-tune only	CVPR'22	19.1	0.257	0.561	0.348	15.40
BBDM	CVPR'23	76.6	0.270	0.568	0.352	15.34
ControlNet	ICCV'23	50.4	0.307	0.604	0.297	13.42
HI-Diff	NeurIPS'23	324.3	0.539	0.692	0.457	17.10
ResShift	NeurIPS'23	140.2	0.334	0.607	0.217	14.20
StegoGAN ⚠	CVPR'24	106.8	0.384	0.658	0.254	12.96
SAR-to-EO
Conditional Diffusion	GRSL'24	88.6	0.355	0.730	0.213	11.55
cBBDM	GRSL'25	50.6	0.246	0.539	0.372	16.02
E3Diff	GRSL'25	47.8	0.278	0.530	0.302	16.44
C-DiffSET	TCSVT'26	19.9	0.233	0.526	0.380	16.92
Ours
ReFlowSET (ours)	—	19.1	0.231	0.534	0.355	16.09

SAR2Opt — n = 627, 512 px (centre crops of the 600 px tiles)

Method	Venue	FID↓	DISTS↓	LPIPS↓	SSIM↑	PSNR↑
General image-to-image
pix2pix	CVPR'17	261.9	0.347	0.657	0.199	13.39
CycleGAN ⚠	ICCV'17	143.5	0.330	0.650	0.178	12.90
pix2pixHD	CVPR'18	146.3	0.283	0.567	0.268	15.95
SPADE	CVPR'19	142.5	0.265	0.597	0.234	14.47
DDPM (SR3-class)	TPAMI'22	122.5	0.295	0.610	0.313	13.65
SD2.1 fine-tune only	CVPR'22	71.8	0.211 †	0.541	0.293	16.24
BBDM	CVPR'23	143.1	0.290	0.590	0.276	15.29
ControlNet	ICCV'23	140.5	0.350	0.643	0.217	11.73
HI-Diff	NeurIPS'23	319.8	0.473	0.692	0.384	17.36
ResShift	NeurIPS'23	141.7	0.304	0.597	0.177	14.31
StegoGAN ⚠	CVPR'24	150.1	0.347	0.655	0.158	12.47
SAR-to-EO
Conditional Diffusion	GRSL'24	211.8	0.415	0.686	0.248	12.48
cBBDM	GRSL'25	222.3	0.377	0.571	0.361	17.05
E3Diff	GRSL'25	104.7	0.232	0.529	0.249	16.09
C-DiffSET	TCSVT'26	78.1	0.214 †	0.529	0.314	16.81
Ours
ReFlowSET (ours)	—	66.3	0.185	0.522	0.287	16.06

⚠ fails the input-copy audit — see Identity collapse below. † these two DISTS values were re-measured. The image dumps for these two SAR2Opt cells were regenerated after their last extended-metrics pass, and that pass's resume check keys on the number of images rather than on their content, so it skipped them and their stored DISTS went stale. Both were re-measured on the current dumps at full n=627 and the table prints the re-measurement (SD2.1-FT 0.212 -> 0.211; C-DiffSET 0.2141 -> 0.2136, unchanged at three decimals). The harness was validated on a cell that is not stale, whose DISTS it reproduces to 6e-6. FID, LPIPS, SSIM and PSNR come from a pass that rescores every cell on every run and were never affected.

Read before you compare these numbers with a paper

These numbers are not comparable with the source papers, and are not meant to be. Three reasons, all deliberate:

1. Budgets are matched across the benchmark, not to each method's own recipe. Several methods are trained here at a fraction of their published schedule — HI-Diff, for instance, is trained for 50,000 updates against an upstream recipe of 600,000, with progressive resizing disabled. Where we departed from a published recipe, the method's own card says so and says by how much.
2. Budgets are quoted in optimizer updates, never epochs. The two datasets differ by more than tenfold in size (16,001 versus 1,450 training pairs), so an epoch count means something completely different on each. Comparing unpaired methods by epochs produced two successive wrong conclusions in our own work before we changed the unit.
3. One evaluation protocol for everyone. Same test items, same resolution, same metric implementations, same ground truth. SAR2Opt is centre-cropped to 512 px and never resized, in training and in evaluation alike.

The LPIPS convention

LPIPS here is LPIPS-VGG on inputs mapped to [-1, 1] before the network — lpips_vgg(x * 2 - 1, y * 2 - 1) with the library's default normalize=False. The other convention, which several released evaluators use, hands the network [0, 1] inputs unscaled (still normalize=False) and produces a systematically lower number — about 0.05 lower — on the same images. Note that [0, 1] inputs with normalize=True is the same thing as ours, not the other one; only the flag differs. Both are called "LPIPS" in the literature. If you compare a number from this table against one from a paper, check which convention that paper used first; otherwise the comparison is meaningless. The same care applies in reverse to anyone quoting these numbers.

PSNR and SSIM are per-image torchmetrics with data_range=1. FID is pytorch-fid against the size-matched ground-truth test set. DISTS is the standard implementation on the same pairs. Where the generated size and the ground-truth size differ, the ground truth is centre-cropped to match.

Identity collapse

Four of the thirty cells fail an input-copy audit: CycleGAN and StegoGAN, on both datasets. Their outputs are closer to the SAR image they were given than to the EO image they were asked for — mean |output − SAR| < mean |output − ground truth| — which is the signature of a model that has learned to pass its input through rather than translate it.

cell	ratio (SAR distance ÷ GT distance)	verdict
QXS-SAROPT CycleGAN	0.847	COLLAPSED
QXS-SAROPT StegoGAN	0.919	COLLAPSED
SAR2Opt CycleGAN	0.795	COLLAPSED
SAR2Opt StegoGAN	0.758	COLLAPSED

Their metric values are reported unchanged and marked in place. They are what the released implementation does at its own published protocol, and a benchmark that quietly substitutes a better-behaved variant is not reporting the method. But they should not be read as translation quality. No other cell on either dataset collapses — in particular DDPM, Conditional Diffusion and BBDM do not collapse here, whatever they may do elsewhere.

The companion leak audit is clean: no cell on either dataset shows a generation that matches its own ground truth anomalously better than it matches other images' ground truth. Per-cell numbers are on each method's card.

One oracle you must not use. StegoGAN's fake_B output is computed as netG_A(real_A, latent(real_B)) — conditioned on the ground truth, and impossible to produce at deployment. Only fake_B_clean = netG_A(real_A) is a prediction, and it is what the StegoGAN rows above report. Any published StegoGAN number should say which of the two it is.

Layout

<dataset>/<method>/<weight files> + README.md
dataset ∈ { qxs-saropt, sar2opt }
licenses/          third-party licence texts, one per method

Every one of the thirty folders carries its own README.md with that cell's architecture, the file-by-file contents, the training budget in optimizer updates, the measured metrics, a copy-pasteable load-and-translate snippet, the traps specific to that method, and its licence stated factually.

method	folder	files	upstream code
pix2pix	pix2pix/	net_G.pth	junyanz/pytorch-CycleGAN-and-pix2pix
CycleGAN	cyclegan/	net_G_A.pth (SAR→EO), net_G_B.pth	junyanz/pytorch-CycleGAN-and-pix2pix
pix2pixHD	p2phd/	net_G.pth	NVIDIA/pix2pixHD
SPADE	spade/	net_G.pth	NVlabs/SPADE
DDPM (SR3-class)	ddpm/	gen.pth	DeepSARRS/E3Diff, stage 1
SD2.1 fine-tune only	sd21ft/	diffusion_pytorch_model.safetensors, config.json	KAIST-VICLab/C-DiffSET, stage 1, on SD 2.1-base
BBDM	bbdm/	last_model.pth	xuekt98/BBDM
ControlNet	controlnet/	diffusion_pytorch_model.safetensors, config.json	huggingface/diffusers train_controlnet.py, on SD 2.1-base
HI-Diff	hidiff/	S1_*.pth ×2, S2_*.pth ×3 (only the three S2_ files are needed at inference)	zhengchen1999/HI-Diff
ResShift	resshift/	ema_model.pth	zsyOAOA/ResShift
StegoGAN	stegogan/	net_G_A.pth (SAR→EO), net_G_B.pth	sian-wusidi/StegoGAN
Conditional Diffusion	conddiff/	ema_final.pt	Coordi777/Conditional-Diffusion-for-SAR-to-Optical-Image-Translation
cBBDM	cbbdm/	last_model.pth	egshkim/ConditionalBBDM-for-VHR-SAR-to-Optical
E3Diff	e3diff/	gen.pth	DeepSARRS/E3Diff, stage 2
C-DiffSET	cdiffset/	diffusion_pytorch_model.safetensors, config.json	KAIST-VICLab/C-DiffSET, stage 2

Three checkpoints these weights need and this repository does not carry

• BBDM and cBBDM need the CompVis latent-diffusion vq-f4 VQGAN (model.ckpt, 756,175,527 bytes) at the path named in their config — at construction time, even though the same weights are also inside last_model.pth. MIT.
• ResShift needs the CompVis vq-f4 autoencoder (221,364,711 bytes). MIT.
• SD2.1-FT, ControlNet and C-DiffSET need Stable Diffusion 2.1-base for the frozen VAE, text encoder, tokenizer and scheduler. Their cards name the exact repository we loaded.
• DDPM and E3Diff need no extra checkpoint, but do need PPB-despeckled and Canny condition images computed from each SAR input; neither can be run from a SAR file alone.

Licences

A single repository-wide licence tag would be wrong for this collection. The fifteen methods fall into three tiers, and the restrictive terms in tiers B and C travel with the individual cells. Each method's card states its position in full; the licence texts are in licenses/.

method	upstream licence	redistribution	what propagates
pix2pix	multi-part BSD (CycleGAN / pix2pix / DCGAN notices)	✅ with the notice	no-endorsement (DCGAN part)
CycleGAN	same file as pix2pix	✅ with the notice	same
pix2pixHD	BSD (NVIDIA 2019) + bundled CycleGAN notice	✅ with both notices	none
SPADE	CC BY-NC-SA 4.0	⚠️ non-commercial only	NonCommercial + ShareAlike — this cell is itself distributed under CC BY-NC-SA 4.0
DDPM (SR3-class)	none declared	⚠️ no express grant — published by our decision	unknown; there is no licence to read
SD2.1 fine-tune only	MIT code (C-DiffSET) / Open RAIL++-M weights	✅ under Open RAIL++-M	Attachment A use restrictions
BBDM	MIT	✅	none (needs the external vq-f4 VQGAN, MIT)
ControlNet	Apache-2.0 code (diffusers) / Open RAIL++-M weights	✅ under Open RAIL++-M	Attachment A use restrictions
HI-Diff	Apache-2.0	✅	ship the licence, keep notices, state your modifications
ResShift	S-Lab License 1.0	⚠️ non-commercial only	NonCommercial; no-endorsement
StegoGAN	none declared	⚠️ no express grant — published by our decision	unknown
Conditional Diffusion	none declared	⚠️ no express grant — published by our decision	unknown (the guided-diffusion parts it forks are MIT)
cBBDM	MIT	✅	none (needs the external vq-f4 VQGAN, MIT)
E3Diff	none declared	⚠️ no express grant — published by our decision	unknown
C-DiffSET	MIT code / Open RAIL++-M weights	✅ under Open RAIL++-M	Attachment A use restrictions

The four with no upstream licence

StegoGAN, E3Diff, DDPM (SR3-class) and Conditional Diffusion are derived from upstream code that publishes no licence at all — no LICENSE, LICENCE, COPYING or NOTICE file, no licence section in the README, and no licence declared to GitHub. Under default copyright, all rights remain reserved by those authors and no express permission to redistribute derived work has been granted to us or to you.

We publish these four anyway, so that the benchmark is reproducible end to end, and we say so plainly rather than implying a permission that does not exist. If you intend to redistribute them or build on them, assess that for yourself, and consider asking the upstream authors directly. licenses/NO-UPSTREAM-LICENSE.md records, per method, exactly what we checked and exactly what we found.

Files in licenses/

LICENSE-pix2pix.txt, LICENSE-cyclegan.txt, LICENSE-p2phd.txt, LICENSE-spade.txt, LICENSE-bbdm.txt, LICENSE-cbbdm.txt, LICENSE-hidiff.txt, LICENSE-resshift.txt, LICENSE-cdiffset.txt, LICENSE-sd21ft.txt — one per method, copied verbatim from the upstream repository we trained with. LICENSE-SoftPool-MIT.txt is a vendored dependency of the E3Diff code base and is included because it is genuinely part of what those two cells were trained with; it is not a licence for E3Diff. NO-UPSTREAM-LICENSE.md covers the four methods with no grant. LICENSE-WEIGHTS.md at the repository root is the CreativeML Open RAIL++-M text, including Attachment A, which governs the three Stable-Diffusion-derived cells.

Datasets

Neither corpus is redistributed here, in any form, and neither may be redistributed from here.

• QXS-SAROPT publishes no licence file; its one stated term is a citation requirement — "This paper must be cited when the dataset is used for research purposes" (Huang et al., arXiv:2103.08259). Its distribution is request-gated by the authors. Obtain it from them.
• SAR2Opt ships an MIT LICENSE, Copyright (c) 2021 MarsZhaoYT. The MIT text grants rights in "the Software"; the repository does not name the imagery sources, so that file is best read as covering the authors' packaging and code. Citation requested: Zhao, Celik, Liu and Li, A Comparative Analysis of GAN-based Methods for SAR-to-Optical Image Translation, IEEE GRSL, 2022.
• The optical side of both datasets is Google-Earth-derived. Copyright in that imagery is held by third parties, not by the dataset authors and not by us. No such imagery appears in this repository.

A note on two venue labels

The table above prints the venue strings as they appear in the ReFlowSET paper. Two of them do not match the year in the upstream repositories' own citation blocks, and both refer to the same paper under an early-access versus issue year:

• Conditional Diffusion — the authors' bibtex says 2023, doi:10.1109/LGRS.2023.3337143.
• E3Diff — the authors' bibtex says 2024, doi:10.1109/LGRS.2024.3506566.

Cite by DOI.

Citation

If you use this benchmark or any of these weights, please cite ReFlowSET:

@article{do2026reflowset,
  title   = {ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation},
  author  = {Jeonghyeok Do<sup>1</sup>, Seungchul Lee<sup>2</sup>, Munchurl Kim<sup>1*</sup>},
  journal = {arXiv preprint arXiv:2609.00968},
  year    = {2026}
}

And please cite each method's original paper — every model card carries the reference — and both dataset papers. The QXS-SAROPT citation is required by that dataset's own terms.