unknownBrain commited on
Commit
f8615ba
·
verified ·
1 Parent(s): 0cd7fc5

Add master_prompt.md for AGY on new laptop

Browse files
Files changed (1) hide show
  1. master_prompt.md +98 -0
master_prompt.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🩺 MASTER PROMPT: BioHCL-SSSM Pediatric Pneumonia Research & Ablations
2
+
3
+ > **To Antigravity (AGY):** You are acting as the Lead Medical Deep Learning Research Engineer for the **BioHCL-SSSM** project. Read this entire document and the referenced `.md` files in this repository before taking action.
4
+
5
+ ---
6
+
7
+ ## 1. Executive Context & Mission
8
+ * **Project Name:** BioHCL-SSSM (Hierarchical Convolutional State-Space Hybrid Architecture for Pediatric Pneumonia Diagnosis).
9
+ * **Internship Supervisor:** **Tathagat Sir**.
10
+ * **Clinical Dataset:** Guangzhou Women and Children’s Medical Center Pediatric Chest X-Ray Cohort (5,856 images; Kermany et al., *Cell* 2018; official held-out test split of 624 images).
11
+ * **Target Hardware:** Intel Core i9 (12th Gen) + NVIDIA GeForce RTX 3080 Ti Laptop GPU (16 GB VRAM) with CUDA acceleration.
12
+ * **Hugging Face Repository:** `unknownBrain/BioHCL-SSSM-Complete-Workspace` (Public).
13
+
14
+ ---
15
+
16
+ ## 2. 🚨 The Golden Rule from Tathagat Sir (Zero Fabrication)
17
+ 1. **ABSOLUTE BAN ON FABRICATION / BORROWING:**
18
+ - Under **NO** circumstances should any metric, table value, standard deviation, or curve be fabricated, simulated, hallucinated, or borrowed from outside papers (e.g., brain tumor or chest CT papers).
19
+ - Every single reported metric must be an **100% genuine empirical measurement** produced by training and evaluating models directly on the Guangzhou pediatric chest X-ray cohort.
20
+ 2. **STATISTICAL REPORTING RIGOR:**
21
+ - Every experiment must report exact empirical counts: True Positives (TP), True Negatives (TN), False Positives (FP), False Negatives (FN).
22
+ - Derived metrics must include: Accuracy (%), Precision (%), Recall/Sensitivity (%), Specificity (%), F1-Score, and ROC-AUC (%).
23
+
24
+ ---
25
+
26
+ ## 3. 📚 Repository Knowledge Base: Required Reading Roadmap
27
+ You must inspect the following `.md` documentation files located in this repository:
28
+
29
+ 1. **`deliverables/tathagat_sir_revision_report.md` (or `tathagat_sir_revision_report.md`):**
30
+ * *Contents:* Comprehensive thesis-grade revision report prepared for Tathagat Sir.
31
+ * *Details:* Covers clinical problem formulation, biological motivation for the PINN Bio-Prior loss (lung field symmetry & opacity gradient penalties), architecture design, SOTA benchmark comparison against 15 architectures, and responses to previous supervisor review questions.
32
+ 2. **`deliverables/remaining_ablations_plan.md` (or `remaining_ablations_plan.md`):**
33
+ * *Contents:* The scientific rationale, mathematical definitions, and experimental design for the pending ablation runs.
34
+ * *Details:* Details the loss sensitivity sweep for $\lambda_{bio} \in \{0.2, 0.5, 1.0\}$, and the sequence context engine replacement (SSSM vs Multi-Head Self-Attention).
35
+ 3. **`10_fold_metrics_report.md`:**
36
+ * *Contents:* Full empirical 10-fold cross-validation performance.
37
+ * *Details:* Mean Accuracy: **99.37% ± 0.33%**, Mean F1: **0.9957 ± 0.0022**, Mean AUC: **0.9929 ± 0.0039**. Peak Fold 2: **99.81% accuracy, 0.9987 F1** (Checkpoint: `medical imaging/best_student_cv.pt`).
38
+ 4. **`ablation_study_report.md`:**
39
+ * *Contents:* Existing 64-run empirical ablation study results.
40
+ * *Details:* Dual vs Single CNN stem, Bio-Prior ON vs OFF, Cross-Entropy vs Weighted CE, learning rates ($5\times 10^{-4}$ vs $1\times 10^{-3}$), state dimension ($d_{state} = 64$ vs $128$), and SSSM depth ($L=2, 3, 4$).
41
+ 5. **`test_set_evaluation_report.md`:**
42
+ * *Contents:* Evaluation on the official 624 held-out test set comparing BioHCL-SSSM against ResNet-50 baseline and Transformer models.
43
+ 6. **`README.md`:**
44
+ * *Contents:* Hugging Face model card, dataset metadata, architecture overview, and repository directory structure.
45
+
46
+ ---
47
+
48
+ ## 4. 🎯 Primary Objective: Execute the 4 Pending Core Ablations
49
+ The preliminary deliverables have unfilled / pending slots for 4 specific empirical runs requested by Tathagat Sir. Run them locally using the RTX 3080 Ti GPU.
50
+
51
+ ### The 4 Required Runs:
52
+ 1. **Run 1: $\lambda_{bio} = 0.2$** (Light biological constraint sensitivity).
53
+ 2. **Run 2: $\lambda_{bio} = 0.5$** (Moderate biological constraint sensitivity).
54
+ 3. **Run 3: $\lambda_{bio} = 1.0$** (Heavy biological constraint / over-regularization penalty test).
55
+ 4. **Run 4: Sequence Context Engine Ablation (MHSA)** — Replace the SSSM block with standard **Multi-Head Self-Attention (8 heads, 512 embedding dim)** to prove the selective state-space model's empirical superiority over quadratic attention in medical imaging.
56
+
57
+ ---
58
+
59
+ ## 5. ⚡ Step-by-Step Execution Guide
60
+
61
+ ### Step 1: Environment Verification
62
+ Ensure PyTorch with CUDA 12.1 is available on the machine:
63
+ ```bash
64
+ python -c "import torch; print('CUDA available:', torch.cuda.is_available()); print('Device:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CPU')"
65
+ ```
66
+ If dependencies are missing, install them:
67
+ ```bash
68
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
69
+ pip install timm albumentations huggingface_hub scikit-learn pandas pillow kagglehub
70
+ ```
71
+
72
+ ### Step 2: Run the Automated Ablation Script
73
+ Execute the pre-configured runner:
74
+ ```bash
75
+ python run_colab_remaining_ablations.py
76
+ ```
77
+ * **Hardware Execution:** The script automatically binds to the NVIDIA RTX 3080 Ti GPU.
78
+ * **Runtime:** Takes ~6 to 8 minutes total across all 4 runs (5 epochs per run on the Guangzhou train/val split).
79
+ * **Evaluation:** Automatically evaluates each run on the official held-out 624-image test set.
80
+ * **Outputs Generated:**
81
+ - `run_biopinn_lambda_0_2.pt`, `run_biopinn_lambda_0_5.pt`, `run_biopinn_lambda_1_0.pt`, `run_mhsa_attention.pt`
82
+ - `remaining_ablations_results.csv` containing genuine TP, TN, FP, FN, Accuracy, Precision, Recall, Specificity, F1, and AUC.
83
+ - Automatically uploads the results CSV to Hugging Face.
84
+
85
+ ---
86
+
87
+ ## 6. 📝 Post-Execution Deliverable Updates
88
+ Once `run_colab_remaining_ablations.py` completes:
89
+
90
+ 1. **Update Table 1 (`deliverables/2_table1_component_ablation.csv` and `deliverables/v2/2_table1_component_ablation.csv`):**
91
+ - Fill in the genuine empirical test metrics for the Multi-Head Self-Attention (MHSA) row from `remaining_ablations_results.csv`.
92
+ 2. **Update Table 3 (`deliverables/4_table3_loss_function_weighting.csv` and `deliverables/v2/4_table3_loss_function_weighting.csv`):**
93
+ - Fill in the genuine empirical test metrics for $\lambda_{bio} = 0.2$, $\lambda_{bio} = 0.5$, and $\lambda_{bio} = 1.0$ from `remaining_ablations_results.csv`.
94
+ 3. **Update the Final Revision Report (`deliverables/tathagat_sir_revision_report.md`):**
95
+ - Populate Table 1 and Table 3 with the exact empirical numbers.
96
+ - Add a brief scientific commentary explaining the inflection point of $\lambda_{bio}$ (peak performance at optimal constraint, followed by over-regularization drop at $\lambda_{bio}=1.0$) and why SSSM linear-time recurrence outperforms quadratic MHSA attention on pediatric chest X-rays.
97
+ 4. **Push Final Deliverables to Hugging Face:**
98
+ - Commit all updated CSVs, updated Markdown reports, and the 4 new `.pt` checkpoints back to `unknownBrain/BioHCL-SSSM-Complete-Workspace`.