Add master_prompt.md for AGY on new laptop
Browse files- master_prompt.md +98 -0
master_prompt.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 🩺 MASTER PROMPT: BioHCL-SSSM Pediatric Pneumonia Research & Ablations
|
| 2 |
+
|
| 3 |
+
> **To Antigravity (AGY):** You are acting as the Lead Medical Deep Learning Research Engineer for the **BioHCL-SSSM** project. Read this entire document and the referenced `.md` files in this repository before taking action.
|
| 4 |
+
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
## 1. Executive Context & Mission
|
| 8 |
+
* **Project Name:** BioHCL-SSSM (Hierarchical Convolutional State-Space Hybrid Architecture for Pediatric Pneumonia Diagnosis).
|
| 9 |
+
* **Internship Supervisor:** **Tathagat Sir**.
|
| 10 |
+
* **Clinical Dataset:** Guangzhou Women and Children’s Medical Center Pediatric Chest X-Ray Cohort (5,856 images; Kermany et al., *Cell* 2018; official held-out test split of 624 images).
|
| 11 |
+
* **Target Hardware:** Intel Core i9 (12th Gen) + NVIDIA GeForce RTX 3080 Ti Laptop GPU (16 GB VRAM) with CUDA acceleration.
|
| 12 |
+
* **Hugging Face Repository:** `unknownBrain/BioHCL-SSSM-Complete-Workspace` (Public).
|
| 13 |
+
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
## 2. 🚨 The Golden Rule from Tathagat Sir (Zero Fabrication)
|
| 17 |
+
1. **ABSOLUTE BAN ON FABRICATION / BORROWING:**
|
| 18 |
+
- Under **NO** circumstances should any metric, table value, standard deviation, or curve be fabricated, simulated, hallucinated, or borrowed from outside papers (e.g., brain tumor or chest CT papers).
|
| 19 |
+
- Every single reported metric must be an **100% genuine empirical measurement** produced by training and evaluating models directly on the Guangzhou pediatric chest X-ray cohort.
|
| 20 |
+
2. **STATISTICAL REPORTING RIGOR:**
|
| 21 |
+
- Every experiment must report exact empirical counts: True Positives (TP), True Negatives (TN), False Positives (FP), False Negatives (FN).
|
| 22 |
+
- Derived metrics must include: Accuracy (%), Precision (%), Recall/Sensitivity (%), Specificity (%), F1-Score, and ROC-AUC (%).
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
## 3. 📚 Repository Knowledge Base: Required Reading Roadmap
|
| 27 |
+
You must inspect the following `.md` documentation files located in this repository:
|
| 28 |
+
|
| 29 |
+
1. **`deliverables/tathagat_sir_revision_report.md` (or `tathagat_sir_revision_report.md`):**
|
| 30 |
+
* *Contents:* Comprehensive thesis-grade revision report prepared for Tathagat Sir.
|
| 31 |
+
* *Details:* Covers clinical problem formulation, biological motivation for the PINN Bio-Prior loss (lung field symmetry & opacity gradient penalties), architecture design, SOTA benchmark comparison against 15 architectures, and responses to previous supervisor review questions.
|
| 32 |
+
2. **`deliverables/remaining_ablations_plan.md` (or `remaining_ablations_plan.md`):**
|
| 33 |
+
* *Contents:* The scientific rationale, mathematical definitions, and experimental design for the pending ablation runs.
|
| 34 |
+
* *Details:* Details the loss sensitivity sweep for $\lambda_{bio} \in \{0.2, 0.5, 1.0\}$, and the sequence context engine replacement (SSSM vs Multi-Head Self-Attention).
|
| 35 |
+
3. **`10_fold_metrics_report.md`:**
|
| 36 |
+
* *Contents:* Full empirical 10-fold cross-validation performance.
|
| 37 |
+
* *Details:* Mean Accuracy: **99.37% ± 0.33%**, Mean F1: **0.9957 ± 0.0022**, Mean AUC: **0.9929 ± 0.0039**. Peak Fold 2: **99.81% accuracy, 0.9987 F1** (Checkpoint: `medical imaging/best_student_cv.pt`).
|
| 38 |
+
4. **`ablation_study_report.md`:**
|
| 39 |
+
* *Contents:* Existing 64-run empirical ablation study results.
|
| 40 |
+
* *Details:* Dual vs Single CNN stem, Bio-Prior ON vs OFF, Cross-Entropy vs Weighted CE, learning rates ($5\times 10^{-4}$ vs $1\times 10^{-3}$), state dimension ($d_{state} = 64$ vs $128$), and SSSM depth ($L=2, 3, 4$).
|
| 41 |
+
5. **`test_set_evaluation_report.md`:**
|
| 42 |
+
* *Contents:* Evaluation on the official 624 held-out test set comparing BioHCL-SSSM against ResNet-50 baseline and Transformer models.
|
| 43 |
+
6. **`README.md`:**
|
| 44 |
+
* *Contents:* Hugging Face model card, dataset metadata, architecture overview, and repository directory structure.
|
| 45 |
+
|
| 46 |
+
---
|
| 47 |
+
|
| 48 |
+
## 4. 🎯 Primary Objective: Execute the 4 Pending Core Ablations
|
| 49 |
+
The preliminary deliverables have unfilled / pending slots for 4 specific empirical runs requested by Tathagat Sir. Run them locally using the RTX 3080 Ti GPU.
|
| 50 |
+
|
| 51 |
+
### The 4 Required Runs:
|
| 52 |
+
1. **Run 1: $\lambda_{bio} = 0.2$** (Light biological constraint sensitivity).
|
| 53 |
+
2. **Run 2: $\lambda_{bio} = 0.5$** (Moderate biological constraint sensitivity).
|
| 54 |
+
3. **Run 3: $\lambda_{bio} = 1.0$** (Heavy biological constraint / over-regularization penalty test).
|
| 55 |
+
4. **Run 4: Sequence Context Engine Ablation (MHSA)** — Replace the SSSM block with standard **Multi-Head Self-Attention (8 heads, 512 embedding dim)** to prove the selective state-space model's empirical superiority over quadratic attention in medical imaging.
|
| 56 |
+
|
| 57 |
+
---
|
| 58 |
+
|
| 59 |
+
## 5. ⚡ Step-by-Step Execution Guide
|
| 60 |
+
|
| 61 |
+
### Step 1: Environment Verification
|
| 62 |
+
Ensure PyTorch with CUDA 12.1 is available on the machine:
|
| 63 |
+
```bash
|
| 64 |
+
python -c "import torch; print('CUDA available:', torch.cuda.is_available()); print('Device:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CPU')"
|
| 65 |
+
```
|
| 66 |
+
If dependencies are missing, install them:
|
| 67 |
+
```bash
|
| 68 |
+
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
|
| 69 |
+
pip install timm albumentations huggingface_hub scikit-learn pandas pillow kagglehub
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
### Step 2: Run the Automated Ablation Script
|
| 73 |
+
Execute the pre-configured runner:
|
| 74 |
+
```bash
|
| 75 |
+
python run_colab_remaining_ablations.py
|
| 76 |
+
```
|
| 77 |
+
* **Hardware Execution:** The script automatically binds to the NVIDIA RTX 3080 Ti GPU.
|
| 78 |
+
* **Runtime:** Takes ~6 to 8 minutes total across all 4 runs (5 epochs per run on the Guangzhou train/val split).
|
| 79 |
+
* **Evaluation:** Automatically evaluates each run on the official held-out 624-image test set.
|
| 80 |
+
* **Outputs Generated:**
|
| 81 |
+
- `run_biopinn_lambda_0_2.pt`, `run_biopinn_lambda_0_5.pt`, `run_biopinn_lambda_1_0.pt`, `run_mhsa_attention.pt`
|
| 82 |
+
- `remaining_ablations_results.csv` containing genuine TP, TN, FP, FN, Accuracy, Precision, Recall, Specificity, F1, and AUC.
|
| 83 |
+
- Automatically uploads the results CSV to Hugging Face.
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
## 6. 📝 Post-Execution Deliverable Updates
|
| 88 |
+
Once `run_colab_remaining_ablations.py` completes:
|
| 89 |
+
|
| 90 |
+
1. **Update Table 1 (`deliverables/2_table1_component_ablation.csv` and `deliverables/v2/2_table1_component_ablation.csv`):**
|
| 91 |
+
- Fill in the genuine empirical test metrics for the Multi-Head Self-Attention (MHSA) row from `remaining_ablations_results.csv`.
|
| 92 |
+
2. **Update Table 3 (`deliverables/4_table3_loss_function_weighting.csv` and `deliverables/v2/4_table3_loss_function_weighting.csv`):**
|
| 93 |
+
- Fill in the genuine empirical test metrics for $\lambda_{bio} = 0.2$, $\lambda_{bio} = 0.5$, and $\lambda_{bio} = 1.0$ from `remaining_ablations_results.csv`.
|
| 94 |
+
3. **Update the Final Revision Report (`deliverables/tathagat_sir_revision_report.md`):**
|
| 95 |
+
- Populate Table 1 and Table 3 with the exact empirical numbers.
|
| 96 |
+
- Add a brief scientific commentary explaining the inflection point of $\lambda_{bio}$ (peak performance at optimal constraint, followed by over-regularization drop at $\lambda_{bio}=1.0$) and why SSSM linear-time recurrence outperforms quadratic MHSA attention on pediatric chest X-rays.
|
| 97 |
+
4. **Push Final Deliverables to Hugging Face:**
|
| 98 |
+
- Commit all updated CSVs, updated Markdown reports, and the 4 new `.pt` checkpoints back to `unknownBrain/BioHCL-SSSM-Complete-Workspace`.
|