BioHCL-SSSM-Complete-Workspace / ablation_study_report.md
unknownBrain's picture
Complete medical imaging workspace dump (weights, deliverables, papers, notebooks, code, figures)
eded815 verified
|
Raw History Blame Contribute Delete
5.36 kB

BioHCL-SSSM Ablation Study Comprehensive Report

This report presents a structured analysis of all ablation experiments conducted on the BioHCL-SSSM architecture for Chest X-Ray Pneumonia diagnosis. All values are directly synthesized from the empirical runs logged in ablation.csv, layerwise_ablation_computational.csv, and benchmark_master.csv.


1. Architectural & Non-Parametric Ablations

Evaluates the necessity of the Dual-Chain feature extraction pipeline and the domain-specific Biological Physics-Informed Prior.

Experiment Variant Configuration Details Params (M) GFLOPs Latency (ms) Val Acc (%) Val F1 $\Delta$ F1 vs Full Key Finding
Dual-Chain + Bio-Prior (Full) chain_mode=dual, bio_prior=True, blocks=2, d_state=64, lr=5e-4, loss=ce 12.15 7.74 41.50 97.61% 0.9841 0.0000 Optimal feature fusion; combines local texture with high-level anatomical semantics.
Without Biological Prior chain_mode=dual, bio_prior=False, blocks=2, d_state=64, lr=5e-4, loss=ce 12.15 7.74 41.30 87.30% 0.9212 -0.0629 -6.29% drop in F1; prior is critical for regularizing ambiguous infiltrative opacities.
Single-Chain (CNN only) chain_mode=single_cnn, bio_prior=True, blocks=2, d_state=64, lr=5e-4, loss=ce 5.12 3.82 22.40 67.62% 0.7932 -0.1909 -19.09% drop in F1; local CNN alone lacks long-range contextual receptive field.
Single-Chain (No Bio-Prior) chain_mode=single_cnn, bio_prior=False, blocks=2, d_state=64, lr=5e-4, loss=ce 5.12 3.82 22.30 74.21% 0.8520 -0.1321 Fails to reach clinical viability without dense global sequence routing.

2. Hyperparameter Ablations

Analyzes sensitivity across optimizer learning rates and internal SSSM state vector dimensions ($d_{state}$).

Hyperparameter Variant Tested Configuration Details Params (M) GFLOPs Latency (ms) Val Acc (%) Val F1 Finding & Analysis
Learning Rate $\text{lr} = 5 \times 10^{-4}$ (Optimal) blocks=4, d_state=64, loss=ce 15.88 9.37 56.70 96.56% 0.9769 Stable zero-order hold (ZOH) discretization for SSSM matrices.
Learning Rate $\text{lr} = 1 \times 10^{-3}$ (High) blocks=4, d_state=64, loss=ce 15.88 9.37 56.70 89.88% 0.9311 F1 dropped by 4.58%; higher lr induces gradient oscillations in recursive scans.
State Dimension $d_{state} = 64$ (Standard) blocks=4, lr=5e-4, loss=ce 15.88 9.37 56.70 96.56% 0.9769 Balanced sequence capacity and compute budget.
State Dimension $d_{state} = 128$ (Expanded) blocks=4, lr=5e-4, loss=ce 16.41 9.72 61.20 96.66% 0.9777 Marginal gain (+0.08% F1) at the cost of +3.5% FLOPs and +4.5ms latency.

3. Loss-Wise Ablations

Compares Standard Cross-Entropy (ce) against Class-Weighted Cross-Entropy (weighted_ce).

Setting Loss Function Configuration Details Val Acc (%) Val F1 Clinical Trade-off & Behavior
Pair A ($d_{state}=64$) Standard CE (ce) blocks=2, d_state=64, lr=5e-4 97.61% 0.9841 Higher overall harmonic mean and accuracy across classes.
Weighted CE (weighted_ce) blocks=2, d_state=64, lr=5e-4 80.32% 0.8535 Heavily penalizes dominant class, lowering overall precision.
Pair B ($d_{state}=128$) Standard CE (ce) blocks=4, d_state=128, lr=5e-4 96.66% 0.9777 Smooth convergence on held-out distributions.
Weighted CE (weighted_ce) blocks=4, d_state=128, lr=5e-4 86.25% 0.8979 Induces conservative thresholding; reduces false negatives at the cost of false positives.

4. Layer-Wise Depth & Computational Complexity Ablation

Profiles scaling performance from 1 SSSM Block up to 6 SSSM Blocks to determine optimal network depth.

SSSM Block Depth Parameters (M) GFLOPs Train Time / Epoch (s) Inference Latency (ms) Val Accuracy (%) Val F1 Architectural Status
1 Block 10.28 6.93 94.20 34.80 91.40% 0.9320 Under-parameterized; shallow receptive field.
2 Blocks 12.15 7.74 114.80 41.50 97.61% 0.9841 High efficiency, strong baseline.
3 Blocks 14.01 8.56 136.37 49.06 99.81% 0.9987 Optimal Pareto Peak (Best Model Checkpoint)
4 Blocks 15.88 9.37 158.10 56.70 96.66% 0.9777 Diminishing returns; over-parameterization begins.
6 Blocks 19.61 10.99 201.50 72.30 95.20% 0.9650 Severe latency overhead (+47%) with reduced validation F1.

All summary data is saved to ablation_study_summary.csv and layerwise_ablation_computational.csv.