File size: 5,359 Bytes
dda557a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | # Project Progress Log
## Phase 0 — Scaffolding (COMPLETED 2026-05-14)
- Folder structure: `data/`, `model/`, `backend/`, `frontend/`.
- Root files: `README.md`, `.gitignore`, `PROGRESS.md`, `docker-compose.yml`.
- Stack decision: **PyTorch 2.12 + CUDA 13.0** (Python 3.13 venv at `./venv`).
- GPU verified: NVIDIA GeForce GTX 1650 with Max-Q Design.
## Phase 1 — Data Layer (COMPLETED 2026-05-14)
- `data/download_data.py` wraps the Kaggle CLI to pull
`masoudnickparvar/brain-tumor-mri-dataset`.
- Dataset extracted to `data/raw/{Training,Testing}/`.
- **Class distribution (perfectly balanced):**
- Training: glioma 1400, meningioma 1400, notumor 1400, pituitary 1400 (5,600 total)
- Testing: glioma 400, meningioma 400, notumor 400, pituitary 400 (1,600 total)
- No class-imbalance handling needed.
## Phase 2 — Model Training (COMPLETED · v2)
### v1 baseline (deprecated, kept under `*_v1.*` filenames)
- Architecture: EfficientNet-B3 + heavy head (Dropout → 512+BN → 256 → 4) at 224×224.
- Result: **Test acc 84.00%**, glioma F1=0.753, val_acc > train_acc through training
(over-regularized head + low-res input → underfit, especially on glioma).
- Files preserved: `brain_tumor_model_v1.pth`, `metrics_v1.json`, `history_v1.json`,
`train.log`.
### v2 changes
- **300×300 input** (matches EfficientNet-B3 pretrain resolution).
- **Minimal head**: `Dropout(0.3) → Linear(1536, 4)`. Backbone-friendly.
- **Phase 1**: head-only, 6 epochs, AdamW lr=1e-3.
- **Phase 2**: unfreeze last 3 feature blocks, 25 epochs, AdamW lr=5e-5 + cosine.
- **Label smoothing 0.05**, mixed-precision (autocast + GradScaler).
- Lighter augmentation (dropped ColorJitter — MRI intensities are diagnostic).
- Eval transform unified: Resize(324) → CenterCrop(300).
### v2 results
- **Test accuracy: 95.00%** (+11 pp vs v1).
- Best val accuracy: 98.57% (epoch 19 of fine-tune).
- Per-class F1: glioma=0.903, meningioma=0.939, notumor=0.966, pituitary=0.989.
- Glioma-misclassified-as-notumor (clinically dangerous false negative) dropped 43 → 25.
- Checkpoint: `model/saved/brain_tumor_model.pth` (val_acc 0.9857).
### Confusion matrix (v2 test set, 1600 images)
```
pred:glioma meningioma notumor pituitary
true:glioma 334 40 25 1 (recall 83.5%)
true:meningi. 3 392 1 4 (recall 98.0%)
true:notumor 2 0 398 0 (recall 99.5%)
true:pituitary 1 3 0 396 (recall 99.0%)
```
### Files
- `model/architecture.py` — model definition + freeze/unfreeze helpers.
- `model/train.py` — two-phase training, AMP, early stopping. Output unbuffered.
- `model/gradcam.py` — Grad-CAM hooks on `model.features[-1]`.
- `model/evaluate.py` — test-set per-class metrics + confusion matrix.
## Phase 3 — Backend (CODE READY · SMOKE-TESTED)
- `backend/main.py` — FastAPI app with `/`, `/health`, `/metrics`, `/predict`.
Uses an async `lifespan` to load the predictor on startup.
- `backend/predictor.py` — Loads checkpoint when present, falls back to ImageNet-init
with a warning so the API stays up during development.
- `backend/Dockerfile` — Python 3.12-slim base with libgl/libglib for OpenCV.
- Verified: lifespan starts, full `predict()` returns class + probs + base64 Grad-CAM.
## Phase 4 — Frontend (CODE READY · BUILD VERIFIED)
- Vite 6 + React 18 + Tailwind 3.4 + Framer Motion 11 + react-dropzone 14.
- `HeroSection` renders an animated neural-network canvas (violet/cyan glowing
nodes + edges, devicePixelRatio-aware).
- `UploadZone` — drag/drop, preview, in-flight spinner overlay.
- `ResultCard` — diagnosis title, severity badge, animated confidence circle,
per-class confidence bars, Grad-CAM side-by-side viewer.
- `TumorInfo` — explains all 4 classes.
- Build size: 338KB JS (108KB gzip), 20KB CSS (4.5KB gzip). Clean build.
## Phase 5 — Docker (CODE READY)
- `backend/Dockerfile`, `frontend/Dockerfile` (multi-stage → nginx).
- `docker-compose.yml` exposes backend on 8000, frontend on 3000.
- Frontend nginx config also proxies `/api/*` → backend container.
---
## Run order (for the user)
```bash
# 1. (one time) put kaggle.json at ~/.kaggle/kaggle.json
./venv/bin/python data/download_data.py
# 2. train (GPU recommended — ~30 min on GTX 1650)
./venv/bin/python model/train.py | tee model/saved/train.log
# 3. evaluate (writes metrics.json)
./venv/bin/python model/evaluate.py
# 4. backend
./venv/bin/uvicorn main:app --reload --app-dir backend --host 0.0.0.0 --port 8000
# 5. frontend
( cd frontend && npm run dev )
# OR everything in containers:
docker compose up --build
```
## Open items
- [x] Add `~/.kaggle/kaggle.json` and run download. ✅ 2026-05-14
- [x] Run v1 training. ✅ 84% test acc — diagnosed underfit on glioma.
- [x] Patch architecture/train for v2 (300px, simpler head, higher FT lr, AMP, buffered output fix).
- [x] Run v2 training. ✅ **Test acc 95.00% · glioma F1 0.903 (was 0.753)**.
- [ ] Boot the FastAPI app + Vite dev server and test the UI end-to-end with a real MRI.
- [ ] Optional: copy 3–4 sample MRIs into `frontend/public/samples/` for demo buttons.
- [ ] Optional: re-run training after the AMP-eval NaN patch if you want clean val_loss
curves in `history.json` (current model is fine; cosmetic only).
|