scripts/EXTERNAL_BASELINES.md

Baselines from TabDiff & fair-tab-diffusion

Both papers benchmark a large set of generators. In MemisisLabs they fall into two tiers.

Tier 1 — available as synthcity plugins (run via the arena, no extra repos)

These enter the arena directly through the synthcity backend once a SYNTHCITY_VENV is set (see scripts/server_arena.py). Install with pip install "synthcity[all]" on the GPU host to get the goggle/great extras.

Baseline (paper)	synthcity plugin
CTGAN	ctgan
TVAE / RTVAE	tvae, rtvae
TabDDPM	ddpm
GOGGLE	goggle
GReaT	great
Normalizing flow	nflow
ARF	arf
DP-GAN / PATE-GAN / ADS-GAN	dpgan, pategan, adsgan
PrivBayes	privbayes
DECAF (fairness)	decaf

Run: python scripts/server_arena.py (uses these by default across all datasets).

Tier 2 — standalone research repos (external experiments)

No pip/synthcity plugin — each is its own repo with its own env. Integrate them the same way as TabDiff: clone + set up, train + sample into a CSV, then score with our metric suite via a small evaluate step (mirror scripts/tabdiff_evaluate.py). This puts them on the shared leaderboard under our TabDiff-standard metrics.

Baseline	Repo	Notes
TabDiff	github.com/MinkaiXu/TabDiff	done — scripts/tabdiff_prepare.py + tabdiff_evaluate.py
TabSyn	github.com/amazon-science/tabsyn	latent-space diffusion
STaSy	github.com/JayoungKim408/STaSy	score-based
CoDi	github.com/ChaejeongLee/CoDi	co-evolving diffusion
FairTabDDPM	github.com/comp-well-org/fair-tab-diffusion	fairness-aware diffusion
FairTabGAN / FairSMOTE	(fair-tab-diffusion args/*/)	fairness-aware baselines

Generic pattern for a Tier-2 baseline

1. Clone + create its env on the lab server.
2. Prepare the dataset in its expected format (adapt scripts/tabdiff_prepare.py if it uses the same Info-JSON layout — TabSyn/CoDi/STaSy/FairTabDDPM all descend from the TabSyn data format).
3. Train + sample to a synthetic CSV.
4. Score + record:

   python scripts/tabdiff_evaluate.py --dataset openml_45040 \
       --synthetic /path/to/<method>_samples/ --record
   

   (change the recorded label in the script, or add a --label <method> flag.)

Fairness metrics

fair-tab-diffusion's fairness = fairlearn demographic_parity_ratio + equalized_odds_ratio — already adopted as the arena's fairness dimension (pipeline/fairness.py), applied to every classification dataset with a sensitive attribute.