kfold/README.md

# 5-Fold Cross-Validation Results — v2 Models

All 9 models were re-trained with the full v2 protocol (CLAHE, RandAugment,
WeightedRandomSampler, MixUp+CutMix, warmup-cosine LR, AMP fp16) on 5
stratified-group folds (pHash-grouped pool, fixed held-out test set of 3,208
images). Results confirm the published headline metrics are robust.

## Summary (sorted by test accuracy)

| Model | Family | Test Acc | 95% CI | Test F1 | ROC-AUC | ECE | Folds |
|---|---|---|---|---|---|---|---|
| Inception-v3 | CNN | 90.18% | [89.24, 91.24] | 92.54% | 0.9930 | 0.0194 | 5 |
| CLIP ViT-B/16 | CLIP | 90.15% | [89.18, 91.24] | 92.83% | 0.9944 | 0.0217 | 5 |
| VGG-19 | CNN | 90.12% | [89.09, 91.12] | 92.59% | 0.9933 | 0.0228 | 5 |
| ResNet-101 | CNN | 90.09% | [89.15, 91.12] | 92.63% | 0.9941 | 0.0243 | 5 |
| DenseNet-121 | CNN | 89.65% | [88.62, 90.71] | 92.29% | 0.9937 | 0.0272 | 5 |
| DINOv2-L | Foundation | 89.61% ± 0.20 | — | 92.27% ± 0.17 | — | — | 5 |
| ResNet-50 | CNN | 89.50% | [88.40, 90.59] | 92.20% | 0.9945 | 0.0339 | 5 |
| Swin-B | Foundation | 87.00% ± 0.26 | — | 90.26% ± 0.18 | — | — | 5 |
| RETFound | Foundation | 83.35% ± 0.23 | — | 87.27% ± 0.20 | — | — | 5 |

## Files

- `kfold_v2_summary.csv` — unified summary of all 9 models
- `cnn_clip/summary.json` — test metrics for all 6 CNN/CLIP models
- `cnn_clip/<model>_kfold.json` — per-fold val metrics (6 models × 5 folds)
- `cnn_clip/mcnemar.json` — pairwise McNemar tests (all p > 0.05 → no
  significant difference between CNN/CLIP architectures)
- `foundation_kfold.json` — aggregated mean ± std for 3 foundation models
- `foundation_fold{0-4}_summary.json` — per-fold summaries

## McNemar Pairwise Tests (CNN/CLIP)

All 15 pairwise McNemar p-values > 0.05. The models perform at statistical
parity on this test set — architecture choice does not significantly alter
prediction errors.

## Protocol

- **CNN/CLIP**: `run_v2_experiments.py --folds 5 --epochs 100 --patience 12`
- **Foundation**: `run_foundation_kfold.py --folds 5 --lp-epochs 20 --ft-epochs 15 --patience 8`
- Split: `splits/holdout_split_augmented.json` (StratifiedGroupKFold, seed=42, pHash threshold=8)
- Test set: 3,208 images, fixed, never used during training or fold selection