SACFlow / docs /AMOS_SHARED13_EXPERIMENTS.md
sathiiii's picture
Add documentation
1734875 verified
|
Raw
History Blame Contribute Delete
3.76 kB

AMOS22 CT->MRI shared-13 experiment plan

This is the first SACFlow-FM implementation/evaluation protocol.

Label protocol

AMOS22 labeled data contains 500 CT and 100 MRI cases. In the local AMOS split, MRI validation labels may only contain labels 0..13; labels 14 and 15 are absent in target_val. For a clean CT->MRI protocol we use a shared-label setting:

  • keep labels 0..13;
  • map labels 14 and 15 to background 0;
  • set num_classes: 14;
  • use target_val for evaluation because target_test labels are not available.

0. Create manifests

From repo root:

AMOS_ROOT=datasets/amos22 bash scripts/amos/create_manifests_shared13.sh

This creates:

data_json/amos_ct2mr.json
data_json/amos_ct2mr_shared13.json
data_json/amos_ct2mr_shared13_sfda.json

Use amos_ct2mr_shared13_sfda.json for all source-free adaptation runs. It removes labels from target_train to avoid target-label leakage.

1. Smoke test

NPROC=1 bash scripts/amos/train_source_ct_smoke.sh

2. Source CT model

NPROC=8 bash scripts/amos/train_source_ct_shared13.sh
bash scripts/amos/eval_source_ct2mri_shared13.sh

3. Export compact source memory

NUM_PASSES=3 bash scripts/amos/export_memory_ct_shared13.sh

4. MRI oracle upper bound

NPROC=8 bash scripts/amos/train_oracle_mri_shared13.sh
python tools/eval.py \
  --config configs/experiments/amos_oracle_mri_shared13.yaml \
  --checkpoint outputs/amos/oracle_mri_shared13/checkpoints/best.pt \
  --split target_val

5. Internal SFDA baselines

NPROC=8 bash scripts/amos/run_selftrain_ct2mri_shared13.sh
NPROC=8 bash scripts/amos/run_peft_ct2mri_shared13.sh
NPROC=8 bash scripts/amos/run_proto_align_ct2mri_shared13.sh

Evaluate:

python tools/eval.py --config configs/experiments/amos_selftrain_ct2mri_shared13.yaml --checkpoint outputs/amos/selftrain_ct2mri_shared13/checkpoints/best.pt --split target_val
python tools/eval.py --config configs/experiments/amos_peft_ct2mri_shared13.yaml --checkpoint outputs/amos/peft_ct2mri_shared13/checkpoints/best.pt --split target_val
python tools/eval.py --config configs/experiments/amos_proto_align_ct2mri_shared13.yaml --checkpoint outputs/amos/proto_align_ct2mri_shared13/checkpoints/best.pt --split target_val

6. SACFlow-FM

NPROC=8 bash scripts/amos/run_sacflow_ct2mri_shared13.sh
bash scripts/amos/eval_sacflow_ct2mri_shared13.sh

7. First ablations

Run these after the main SACFlow-FM run works:

NPROC=8 bash scripts/amos/run_sacflow_linear_ct2mri_shared13.sh
NPROC=8 bash scripts/amos/run_sacflow_whole_feature_ct2mri_shared13.sh
NPROC=8 bash scripts/amos/run_sacflow_random_subspace_ct2mri_shared13.sh

Evaluate:

python tools/eval.py --config configs/experiments/amos_sacflow_linear_ct2mri_shared13.yaml --checkpoint outputs/amos/sacflow_linear_ct2mri_shared13/checkpoints/best.pt --split target_val
python tools/eval.py --config configs/experiments/amos_sacflow_whole_feature_ct2mri_shared13.yaml --checkpoint outputs/amos/sacflow_whole_feature_ct2mri_shared13/checkpoints/best.pt --split target_val
python tools/eval.py --config configs/experiments/amos_sacflow_random_subspace_ct2mri_shared13.yaml --checkpoint outputs/amos/sacflow_random_subspace_ct2mri_shared13/checkpoints/best.pt --split target_val

First results table

Report:

Method Target labels for training? Source images during adaptation? Mean Dice Mean HD95
Source-only CT->MRI No No
MRI oracle Yes N/A
Self-training No No
PEFT-only No No
Proto-align internal No No
SACFlow-linear No No
SACFlow-FM No No