# Setup — Flow-CoPD experiments

Method: **Flow-CoPD** (Contrastive On-Policy Distillation for flow). See `refine-logs/FINAL_PROPOSAL.md` + `refine-logs/EXPERIMENT_PLAN.md`.

## Codebase decision
**Base codebase = Flow-GRPO** (`third_party/flow_grpo`, NeurIPS'25, `github.com/yifan123/flow_grpo`).
Reason: it is the de-facto flow-matching online-RL/OPD codebase (SD3.5-M + LoRA + ODE→SDE sampling), and it already ships the exact pieces our experiment plan needs:

| Our plan needs | Already in flow_grpo |
|---|---|
| positive-only OPD base (A0, = Flow-OPD/DiffusionOPD repro) | `scripts/train_sd3.py` + `config/grpo.py` |
| **Diffusion-DPO control (Block B1)** | `scripts/train_sd3_dpo.py` + `config/dpo.py` |
| reward models (PickScore / aesthetic / CLIP / OCR / ImageReward) | `flow_grpo/{pickscore,aesthetic,clip,ocr,imagereward}_scorer.py` |
| Flow-GRPO-Fast (1–2 step training, single-GPU friendly) | `scripts/train_sd3_fast.py` |
| over-optimization / reward-hacking baseline | `scripts/train_sd3_GRPO_Guard.py` (relevant to our hacking-suppression claim) |

Baselines cloned alongside:
- `third_party/DiffusionNFT` (`NVlabs/DiffusionNFT`) — **Block B2 control** (negative-aware but GRPO-framed, no teacher).

Our method code (to be written) goes in `flow_copd/` — a new `train_sd3_copd.py` + a contrastive-negative loss module, reusing flow_grpo's sampler/reward/teacher-loading.

## Models
| Role | HF id | Where | Notes |
|---|---|---|---|
| Base | `stabilityai/stable-diffusion-3.5-medium` | HF cache (`.hf_cache`) | **GATED — needs your HF token + license** |
| Teacher T1 (preference) | `jieliu/SD3.5M-FlowGRPO-PickScore` | `checkpoints/teachers/pickscore` | public **LoRA**, no training needed |
| Teacher T2 (text/OCR) | `jieliu/SD3.5M-FlowGRPO-Text` | `checkpoints/teachers/text` | public **LoRA**, no training needed |
| Reward: PickScore | `laion/CLIP-ViT-H-14-laion2B-s32B-b79K` + `yuvalkirstain/PickScore_v1` | HF cache | public |
| Reward: aesthetic/CLIP | `openai/clip-vit-large-patch14` (+ repo asset MLP head) | HF cache / repo | public |
| Reward: OCR | PaddleOCR | local cache | installed in `setup_env.sh` |

> 💡 Big win: the Flow-GRPO **task experts double as our teachers** (LoRA adapters), so the ~20–30 GPU·h "teacher prep" in the experiment plan is **eliminated** — just download them.

## Setup order
```bash
# 1. Environment (dedicated conda env; repo pins py3.10.16 / torch2.6.0)
bash scripts/setup_env.sh

# 2a. Public weights (teachers + reward models) — no token needed
bash scripts/download_weights.sh

# 2b. Gated base model — AFTER you accept the license + log in:
#     https://huggingface.co/stabilityai/stable-diffusion-3.5-medium
huggingface-cli login          # or: export HF_TOKEN=hf_xxx
bash scripts/download_weights.sh --all

# 3. Always export this when training (so code finds cached base + reward models):
export HF_HOME=/workspace/Research/UMM/.hf_cache
```

## ⚠️ The one thing only you can do
`stabilityai/stable-diffusion-3.5-medium` is gated. Accept the license and run `huggingface-cli login` (or `export HF_TOKEN=...`). Everything else (code, teachers, reward models, OCR) is public and set up automatically.

## Layout
```
third_party/flow_grpo     # BASE codebase
third_party/DiffusionNFT  # Block B2 baseline
scripts/{setup_env,download_weights}.sh
checkpoints/teachers/{pickscore,text}   # teacher LoRA adapters
.hf_cache/                # base + reward models (HF_HOME)
flow_copd/                # OUR method code (next step via /experiment-bridge)
refine-logs/              # proposal + experiment plan
```