BioHCL-SSSM-Complete-Workspace / test_set_evaluation_report.md
unknownBrain's picture
Complete medical imaging workspace dump (weights, deliverables, papers, notebooks, code, figures)
eded815 verified
|
Raw History Blame Contribute Delete
2.38 kB

Held-Out Sequestered Test Set Evaluation Report

This report presents the diagnostic evaluation on the official held-out test split of the Chest X-Ray Pneumonia dataset ($N = 624$ images: $234$ NORMAL, $390$ PNEUMONIA). These images were strictly sequestered and never seen during cross-validation or hyperparameter selection.


1. Held-Out Test Set Performance Matrix

Model Architecture Test Accuracy (%) Test Precision Test Recall (Sensitivity) Test F1-Score Test ROC-AUC Parameters (M) Parameter Reduction vs Baseline
ResNet-50 Baseline 84.46% 0.8033 0.9949 0.8889 0.9605 23.51 M Baseline (0%)
BioHCL-SSSM (No BioPrior Ablation) 82.37% 0.7811 0.9974 0.8761 0.9506 14.01 M -40.4%
Student BioHCL-SSSM (Full Model) 87.50% 0.8362 0.9949 0.9087 0.9391 14.01 M -40.4% (-9.50M params)

Key Diagnostic Highlights

  • Accuracy Improvement: BioHCL-SSSM surpasses ResNet-50 by +3.04 percentage points ($87.50%$ vs $84.46%$).
  • F1 Improvement: Harmonic mean rises from 0.8889 to 0.9087 ($+0.0198$).
  • Model Footprint: BioHCL-SSSM reduces parameter count by 40.4% ($14.01\text{M}$ vs $23.51\text{M}$).
  • Bio-Prior Efficacy: Ablating the biological prior vector drops test accuracy by 5.13 percentage points ($87.50% \rightarrow 82.37%$) and F1 by -0.0326, verifying that physics-informed biological inductive biases improve test generalization.

2. Test Set Visualizations & Confusion Matrices

### 1. Student BioHCL-SSSM Test Confusion Matrix
![BioHCL-SSSM Test CM](./test_student_biohcl_sssm_cm.png)
<!-- slide -->
### 2. Student BioHCL-SSSM Test ROC Curve
![BioHCL-SSSM Test ROC](./test_student_biohcl_sssm_roc.png)
<!-- slide -->
### 3. Baseline ResNet-50 Test Confusion Matrix
![ResNet-50 Test CM](./test_baseline_resnet50_cm.png)
<!-- slide -->
### 4. Baseline ResNet-50 Test ROC Curve
![ResNet-50 Test ROC](./test_baseline_resnet50_roc.png)
<!-- slide -->
### 5. ROC & Precision-Recall Comparison (Ablation)
![ROC PR Comparison](./test_roc_pr_comparison.png)
<!-- slide -->
### 6. BioPrior Ablated Model Test Confusion Matrix
![BioPrior Ablation Test CM](./test_bioprior_ablation_cm.png)