# SACFlow-FM experiment guide

This guide lists the recommended first set of experiments.

## 0. Environment

```bash
conda env create -f environment.yml
conda activate sacflow
pip install -e .
```

For AMD MI210/ROCm, install the PyTorch ROCm wheel that matches the cluster ROCm version. Example for ROCm 6.2:

```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.2
python -c "import torch; print(torch.__version__, torch.cuda.is_available(), torch.cuda.device_count())"
```

On ROCm, PyTorch still exposes AMD GPUs through the `torch.cuda` API. Use `torchrun` exactly as with CUDA GPU clusters.

## 1. Prepare dataset manifests

The code uses JSON manifests rather than hard-coded dataset layouts.

### MMWHS CT -> MR

```bash
python tools/create_dataset_json.py \
  --name mmwhs_ct2mr \
  --source-images /data/MMWHS/CT/images \
  --source-labels /data/MMWHS/CT/labels \
  --target-images /data/MMWHS/MR/images \
  --target-labels /data/MMWHS/MR/labels \
  --output data_json/mmwhs_ct2mr.json \
  --val-fraction 0.2 \
  --test-fraction 0.2 \
  --seed 1337
```

### MMWHS MR -> CT

Swap source and target folders and write `data_json/mmwhs_mr2ct.json`.

### CrossMoDA ceT1 -> hrT2

```bash
python tools/create_dataset_json.py \
  --name crossmoda_t1_to_t2 \
  --source-images /data/CrossMoDA/ceT1/images \
  --source-labels /data/CrossMoDA/ceT1/labels \
  --target-images /data/CrossMoDA/hrT2/images \
  --target-labels /data/CrossMoDA/hrT2/labels_optional_for_eval \
  --output data_json/crossmoda_t1_to_t2.json
```

If target labels are not available, keep `target-labels` omitted and use a separate held-out validation set only when you have labels.

## 2. Main MMWHS experiment sequence

### A. Source training

```bash
bash scripts/mmwhs/train_source_ct2mr.sh
```

### B. Export compact source memory

```bash
bash scripts/mmwhs/export_memory_ct.sh
```

### C. Source-only target evaluation

```bash
bash scripts/mmwhs/eval_source_ct2mr.sh
```

### D. Oracle target supervised upper bound

```bash
bash scripts/mmwhs/train_oracle_mr.sh
```

### E. Internal SFDA baselines

```bash
bash scripts/mmwhs/run_selftrain_ct2mr.sh
bash scripts/mmwhs/run_peft_ct2mr.sh
bash scripts/mmwhs/run_proto_align_ct2mr.sh
```

### F. SACFlow-FM and key ablations

```bash
bash scripts/mmwhs/run_sacflow_ct2mr.sh
bash scripts/mmwhs/run_sacflow_linear_ct2mr.sh
bash scripts/mmwhs/run_sacflow_whole_feature_ct2mr.sh
bash scripts/mmwhs/run_sacflow_random_subspace_ct2mr.sh
```

## 3. External baselines

For CCRC, ProtoContra, A3-DualUD, HEAL, and RSA, prefer official repositories and run them on the exact same manifest split. Do not compare reported numbers in the main table unless the dataset split, backbone, label set, and metric are exactly identical.

Recommended first external baselines:

1. CCRC on MMWHS.
2. ProtoContra or A3-DualUD on MMWHS.
3. RSA on CrossMoDA.

## 4. W&B

Set:

```bash
export WANDB_PROJECT=SACFlow-FM
export WANDB_ENTITY=<your_entity_or_leave_empty>
wandb login
```

For offline logging:

```bash
export WANDB_MODE=offline
```

## 5. DDP on 8 AMD GPUs

Each script uses `torchrun`. Example:

```bash
torchrun --standalone --nnodes=1 --nproc_per_node=8 tools/train.py --config configs/experiments/mmwhs_sacflow_ct2mr.yaml
```

## 6. Metrics to monitor

Training logs include:

- total loss and per-loss components;
- CE, Dice, entropy, consistency;
- SACFlow FM loss;
- path-state validity weight;
- domain-progress score;
- task KL/logit drift;
- boundary loss;
- velocity magnitude;
- nullspace leakage;
- pseudo-label confidence and class distribution;
- Dice and HD95/ASSD during validation;
- accepted-state rate;
- ETA and iteration speed.