BioHCL-SSSM-Complete-Workspace / test_set_evaluation_report.md
unknownBrain's picture
Complete medical imaging workspace dump (weights, deliverables, papers, notebooks, code, figures)
eded815 verified
|
Raw History Blame Contribute Delete
2.38 kB
# Held-Out Sequestered Test Set Evaluation Report
This report presents the diagnostic evaluation on the official held-out test split of the Chest X-Ray Pneumonia dataset ($N = 624$ images: $234$ NORMAL, $390$ PNEUMONIA). These images were strictly sequestered and never seen during cross-validation or hyperparameter selection.
---
## 1. Held-Out Test Set Performance Matrix
| Model Architecture | Test Accuracy (%) | Test Precision | Test Recall (Sensitivity) | Test F1-Score | Test ROC-AUC | Parameters (M) | Parameter Reduction vs Baseline |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **ResNet-50 Baseline** | 84.46% | 0.8033 | 0.9949 | 0.8889 | 0.9605 | 23.51 M | Baseline (0%) |
| **BioHCL-SSSM (No BioPrior Ablation)** | 82.37% | 0.7811 | **0.9974** | 0.8761 | 0.9506 | 14.01 M | -40.4% |
| **Student BioHCL-SSSM (Full Model)** | **87.50%** | **0.8362** | 0.9949 | **0.9087** | 0.9391 | **14.01 M** | **-40.4% (-9.50M params)** |
### Key Diagnostic Highlights
* **Accuracy Improvement:** BioHCL-SSSM surpasses ResNet-50 by **+3.04 percentage points** ($87.50\%$ vs $84.46\%$).
* **F1 Improvement:** Harmonic mean rises from **0.8889 to 0.9087** ($+0.0198$).
* **Model Footprint:** BioHCL-SSSM reduces parameter count by **40.4%** ($14.01\text{M}$ vs $23.51\text{M}$).
* **Bio-Prior Efficacy:** Ablating the biological prior vector drops test accuracy by **5.13 percentage points** ($87.50\% \rightarrow 82.37\%$) and F1 by **-0.0326**, verifying that physics-informed biological inductive biases improve test generalization.
---
## 2. Test Set Visualizations & Confusion Matrices
````carousel
### 1. Student BioHCL-SSSM Test Confusion Matrix
![BioHCL-SSSM Test CM](./test_student_biohcl_sssm_cm.png)
<!-- slide -->
### 2. Student BioHCL-SSSM Test ROC Curve
![BioHCL-SSSM Test ROC](./test_student_biohcl_sssm_roc.png)
<!-- slide -->
### 3. Baseline ResNet-50 Test Confusion Matrix
![ResNet-50 Test CM](./test_baseline_resnet50_cm.png)
<!-- slide -->
### 4. Baseline ResNet-50 Test ROC Curve
![ResNet-50 Test ROC](./test_baseline_resnet50_roc.png)
<!-- slide -->
### 5. ROC & Precision-Recall Comparison (Ablation)
![ROC PR Comparison](./test_roc_pr_comparison.png)
<!-- slide -->
### 6. BioPrior Ablated Model Test Confusion Matrix
![BioPrior Ablation Test CM](./test_bioprior_ablation_cm.png)
````