Setup — Flow-CoPD experiments
Method: Flow-CoPD (Contrastive On-Policy Distillation for flow). See refine-logs/FINAL_PROPOSAL.md + refine-logs/EXPERIMENT_PLAN.md.
Codebase decision
Base codebase = Flow-GRPO (third_party/flow_grpo, NeurIPS'25, github.com/yifan123/flow_grpo).
Reason: it is the de-facto flow-matching online-RL/OPD codebase (SD3.5-M + LoRA + ODE→SDE sampling), and it already ships the exact pieces our experiment plan needs:
| Our plan needs | Already in flow_grpo |
|---|---|
| positive-only OPD base (A0, = Flow-OPD/DiffusionOPD repro) | scripts/train_sd3.py + config/grpo.py |
| Diffusion-DPO control (Block B1) | scripts/train_sd3_dpo.py + config/dpo.py |
| reward models (PickScore / aesthetic / CLIP / OCR / ImageReward) | flow_grpo/{pickscore,aesthetic,clip,ocr,imagereward}_scorer.py |
| Flow-GRPO-Fast (1–2 step training, single-GPU friendly) | scripts/train_sd3_fast.py |
| over-optimization / reward-hacking baseline | scripts/train_sd3_GRPO_Guard.py (relevant to our hacking-suppression claim) |
Baselines cloned alongside:
third_party/DiffusionNFT(NVlabs/DiffusionNFT) — Block B2 control (negative-aware but GRPO-framed, no teacher).
Our method code (to be written) goes in flow_copd/ — a new train_sd3_copd.py + a contrastive-negative loss module, reusing flow_grpo's sampler/reward/teacher-loading.
Models
| Role | HF id | Where | Notes |
|---|---|---|---|
| Base | stabilityai/stable-diffusion-3.5-medium |
HF cache (.hf_cache) |
GATED — needs your HF token + license |
| Teacher T1 (preference) | jieliu/SD3.5M-FlowGRPO-PickScore |
checkpoints/teachers/pickscore |
public LoRA, no training needed |
| Teacher T2 (text/OCR) | jieliu/SD3.5M-FlowGRPO-Text |
checkpoints/teachers/text |
public LoRA, no training needed |
| Reward: PickScore | laion/CLIP-ViT-H-14-laion2B-s32B-b79K + yuvalkirstain/PickScore_v1 |
HF cache | public |
| Reward: aesthetic/CLIP | openai/clip-vit-large-patch14 (+ repo asset MLP head) |
HF cache / repo | public |
| Reward: OCR | PaddleOCR | local cache | installed in setup_env.sh |
💡 Big win: the Flow-GRPO task experts double as our teachers (LoRA adapters), so the ~20–30 GPU·h "teacher prep" in the experiment plan is eliminated — just download them.
Setup order
# 1. Environment (dedicated conda env; repo pins py3.10.16 / torch2.6.0)
bash scripts/setup_env.sh
# 2a. Public weights (teachers + reward models) — no token needed
bash scripts/download_weights.sh
# 2b. Gated base model — AFTER you accept the license + log in:
# https://huggingface.co/stabilityai/stable-diffusion-3.5-medium
huggingface-cli login # or: export HF_TOKEN=hf_xxx
bash scripts/download_weights.sh --all
# 3. Always export this when training (so code finds cached base + reward models):
export HF_HOME=/workspace/Research/UMM/.hf_cache
⚠️ The one thing only you can do
stabilityai/stable-diffusion-3.5-medium is gated. Accept the license and run huggingface-cli login (or export HF_TOKEN=...). Everything else (code, teachers, reward models, OCR) is public and set up automatically.
Layout
third_party/flow_grpo # BASE codebase
third_party/DiffusionNFT # Block B2 baseline
scripts/{setup_env,download_weights}.sh
checkpoints/teachers/{pickscore,text} # teacher LoRA adapters
.hf_cache/ # base + reward models (HF_HOME)
flow_copd/ # OUR method code (next step via /experiment-bridge)
refine-logs/ # proposal + experiment plan