Relixsx commited on
Commit
3a8534b
Β·
0 Parent(s):

Deploy MedAI backend to Hugging Face Space

Browse files
.dockerignore ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Keep the image small and the build fast β€” never copy these in
2
+ venv/
3
+ .venv/
4
+ __pycache__/
5
+ *.pyc
6
+ *.pyo
7
+ .git/
8
+ .gitignore
9
+ *.bak
10
+ *.ipynb
11
+ notebooks/
12
+
13
+ # Training / data artifacts not needed to serve
14
+ test_images/
15
+ data/
16
+ datasets/
17
+ outputs/
18
+ roi-outputs/
19
+ *.csv
20
+ *.zip
21
+
22
+ # Local frontend (served separately on Truehost)
23
+ frontend/
.gitattributes ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ *.pth filter=lfs diff=lfs merge=lfs -text
2
+ *.png filter=lfs diff=lfs merge=lfs -text
3
+ *.jpg filter=lfs diff=lfs merge=lfs -text
4
+ *.jpeg filter=lfs diff=lfs merge=lfs -text
5
+ *.gif filter=lfs diff=lfs merge=lfs -text
6
+ *.webp filter=lfs diff=lfs merge=lfs -text
.gitignore ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ venv/
2
+ .venv/
3
+ __pycache__/
4
+ *.pyc
5
+ *.bak
6
+ .DS_Store
7
+ test_images/
8
+ data/
9
+ datasets/
10
+ outputs/
11
+ *.zip
12
+ .env
13
+ .test_tmp/
14
+ training_log.csv
Dockerfile ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MedAI backend β€” Hugging Face Spaces (Docker SDK)
2
+ FROM python:3.11-slim
3
+
4
+ # System libraries needed by OpenCV (libgl/libglib) and PyTorch (libgomp)
5
+ RUN apt-get update && apt-get install -y --no-install-recommends \
6
+ libglib2.0-0 libgl1 libgomp1 \
7
+ && rm -rf /var/lib/apt/lists/*
8
+
9
+ # Hugging Face Spaces runs the container as uid 1000
10
+ RUN useradd -m -u 1000 user
11
+ USER user
12
+ ENV HOME=/home/user \
13
+ PATH=/home/user/.local/bin:$PATH \
14
+ PYTHONUNBUFFERED=1 \
15
+ USE_LLM_MODEL=false
16
+ WORKDIR /home/user/app
17
+
18
+ # Install CPU-only PyTorch first (the default CUDA build is ~2 GB larger and unused here)
19
+ RUN pip install --no-cache-dir --upgrade pip && \
20
+ pip install --no-cache-dir \
21
+ torch==2.3.1 torchvision==0.18.1 \
22
+ --index-url https://download.pytorch.org/whl/cpu
23
+
24
+ # Install the rest of the serving dependencies (layer cached unless this file changes)
25
+ COPY --chown=user:user requirements-deploy.txt .
26
+ RUN pip install --no-cache-dir -r requirements-deploy.txt
27
+
28
+ # Copy the application code + model weights
29
+ COPY --chown=user:user . .
30
+
31
+ # HF Spaces expects the app on port 7860
32
+ EXPOSE 7860
33
+ CMD ["uvicorn", "api.main:app", "--host", "0.0.0.0", "--port", "7860"]
README.md ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: MedAI Breast Cancer Platform
3
+ emoji: 🩺
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ license: mit
10
+ ---
11
+
12
+ # MedAI β€” Backend API
13
+
14
+ FastAPI backend for the MedAI dual-modality breast-cancer platform:
15
+
16
+ - **Histopathology** β€” DenseNet-121 fine-tuned on PCam (~88.0% accuracy, 87.5% sensitivity)
17
+ - **Mammography** β€” ensemble of 3Γ— EfficientNet-B4 on RSNA (patient-level AUC 0.8443)
18
+ - **Shared** β€” Grad-CAM heatmaps, audience-aware explanations, Groq-powered chat assistant
19
+
20
+ **Research / educational use only β€” not a medical device and not a substitute for professional diagnosis.**
21
+
22
+ ## Key endpoints
23
+ - `GET /health`
24
+ - `POST /predict`, `POST /explain/visual` β€” histopathology
25
+ - `POST /mammogram/predict`, `POST /mammogram/visual` β€” mammography
26
+ - `POST /chat/stream` β€” streaming chat (Groq)
27
+
28
+ ## Configuration
29
+ Set these as **Space secrets** (Settings β†’ Variables and secrets):
30
+ - `GROQ_API_KEY` β€” required for the chat assistant
31
+ - `CHAT_MODEL` *(optional)* β€” defaults to `openai/gpt-oss-120b`
32
+ - `USE_LLM_MODEL` *(optional)* β€” leave `false` to use the lightweight template explainer
api/main.py ADDED
@@ -0,0 +1,744 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ api/main.py
3
+ ───────────
4
+ FastAPI layer exposing the breast cancer classifier as an HTTP API.
5
+
6
+ Endpoints
7
+ ─────────
8
+ GET /health β€” Liveness check
9
+ POST /predict β€” Single image inference (histopathology)
10
+ POST /predict/batch β€” Multi-image batch inference
11
+ POST /explain β€” Prediction + LLM natural language report
12
+ POST /explain/visual β€” Prediction + Grad-CAM heatmap (base64 PNG)
13
+ POST /chat β€” Conversational follow-up
14
+ POST /mammogram/predict β€” Mammogram inference (3-model ensemble)
15
+ POST /mammogram/explain β€” Mammogram + LLM explanation
16
+ POST /mammogram/visual β€” Mammogram + Grad-CAM + LLM explanation
17
+
18
+ Run locally
19
+ ───────────
20
+ uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload
21
+
22
+ Env vars
23
+ ────────
24
+ WEIGHTS_PATH model/weights.pth (histopathology)
25
+ MAMMO_WEIGHTS_DIR Directory of ensemble members (default: model)
26
+ MAMMO_THRESHOLD Ensemble decision threshold (default: 0.5)
27
+ DEVICE "cuda" | "mps" | "cpu" (default: auto)
28
+ CONFIDENCE_THR Float in [0,1] (default: 0.5)
29
+ USE_LLM_MODEL "true" | "false" (default: false)
30
+ LLM_MODEL_NAME HuggingFace model id (default: flan-t5-base)
31
+ GRADCAM_ALPHA Heatmap blend opacity 0–1 (default: 0.5)
32
+ """
33
+
34
+ from __future__ import annotations
35
+
36
+ import base64
37
+ import io
38
+ import logging
39
+ import math
40
+ import os
41
+ import sys
42
+ from pathlib import Path
43
+ from typing import List, Optional
44
+
45
+ import torch
46
+ from fastapi import FastAPI, File, HTTPException, Query, UploadFile, status
47
+ from fastapi.middleware.cors import CORSMiddleware
48
+ from fastapi.responses import StreamingResponse
49
+ from pydantic import BaseModel, Field
50
+ from PIL import Image
51
+
52
+ # Ensure project root is on PYTHONPATH when running from repo root
53
+ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
54
+
55
+ from model.inference import BreastCancerInferencePipeline
56
+ from model.mammogram_inference import MammogramInferencePipeline
57
+ from model.mammogram_ensemble import MammogramEnsemble
58
+ from explainability.gradcam import GradCAM
59
+ from explainability.mammogram_gradcam import MammogramGradCAM
60
+ from explainability.chat_pipeline import DualModelChatPipeline, create_pipeline
61
+ from explainability.llm_explain import LLMExplainer, ChatEngine
62
+ from explainability.llm_chat import stream_chat, llm_available
63
+
64
+ # ── Logging ───────────────────────────────────────────────────────────────────
65
+ logging.basicConfig(level=logging.INFO)
66
+ logger = logging.getLogger(__name__)
67
+
68
+
69
+ # ── FastAPI app ───────────────────────────────────────────────────────────────
70
+ app = FastAPI(
71
+ title="MedAI β€” Breast Cancer Analysis Platform",
72
+ description=(
73
+ "DenseNet-121 histopathology classifier and 3-model EfficientNet-B4 "
74
+ "mammogram ensemble, with Grad-CAM explainability and LLM explanation. "
75
+ "Research and educational use only. Not a standalone diagnostic tool."
76
+ ),
77
+ version="2.1.0",
78
+ docs_url="/docs",
79
+ redoc_url="/redoc",
80
+ )
81
+
82
+ app.add_middleware(
83
+ CORSMiddleware,
84
+ allow_origins=["*"],
85
+ allow_methods=["GET", "POST"],
86
+ allow_headers=["*"],
87
+ )
88
+
89
+
90
+ # ── Module singletons ─────────────────────────────────────────────────────────
91
+ _pipeline: BreastCancerInferencePipeline | None = None
92
+ _mammo_pipeline: "MammogramEnsemble | MammogramInferencePipeline | None" = None
93
+ _gradcam: GradCAM | None = None
94
+ _llm_explainer: LLMExplainer | None = None
95
+ _chat_engine: ChatEngine | None = None
96
+ _chat_pipeline: DualModelChatPipeline | None = None
97
+
98
+
99
+ @app.on_event("startup")
100
+ def _load_pipeline() -> None:
101
+ global _pipeline, _mammo_pipeline
102
+ weights = os.getenv("WEIGHTS_PATH", "model/weights.pth")
103
+ device = os.getenv("DEVICE", None)
104
+ thr = float(os.getenv("CONFIDENCE_THR", "0.5"))
105
+
106
+ weights_path = Path(weights) if Path(weights).exists() else None
107
+ if weights_path is None:
108
+ logger.warning(
109
+ "weights.pth not found at '%s'. "
110
+ "Running with ImageNet-pretrained backbone only.", weights,
111
+ )
112
+
113
+ _pipeline = BreastCancerInferencePipeline(
114
+ weights_path=weights_path,
115
+ device=device,
116
+ confidence_threshold=thr,
117
+ )
118
+ logger.info("Histopathology pipeline ready on device: %s", _pipeline.device)
119
+
120
+ # ── Mammogram: prefer the 3-model ensemble; fall back to single-view ──────
121
+ mammo_dir = Path(os.getenv("MAMMO_WEIGHTS_DIR", "model"))
122
+ mammo_thr = float(os.getenv("MAMMO_THRESHOLD", "0.5"))
123
+ members = [mammo_dir / f"model_s{s}.pth" for s in (42, 123, 999)]
124
+ present = [p for p in members if p.exists()]
125
+
126
+ if present:
127
+ _mammo_pipeline = MammogramEnsemble(
128
+ weight_paths = present,
129
+ device = device,
130
+ threshold = mammo_thr,
131
+ )
132
+ logger.info(
133
+ "Mammogram ENSEMBLE ready on %s (%d members, threshold=%.2f)",
134
+ _mammo_pipeline.device, len(present), mammo_thr,
135
+ )
136
+ else:
137
+ legacy = mammo_dir / "mammogram_weights.pth"
138
+ _mammo_pipeline = MammogramInferencePipeline(
139
+ weights_path = legacy if legacy.exists() else None,
140
+ device = device,
141
+ )
142
+ logger.warning(
143
+ "Ensemble members not found in '%s'. Falling back to single-view "
144
+ "pipeline (%s).", mammo_dir,
145
+ "trained weights" if legacy.exists() else "ImageNet init",
146
+ )
147
+
148
+ logger.info("Grad-CAM and LLM explainer load on first /explain request.")
149
+
150
+
151
+ def _get_mammo_pipeline():
152
+ if _mammo_pipeline is None:
153
+ raise HTTPException(
154
+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
155
+ detail="Mammogram pipeline not ready.",
156
+ )
157
+ return _mammo_pipeline
158
+
159
+
160
+ def _get_pipeline() -> BreastCancerInferencePipeline:
161
+ if _pipeline is None:
162
+ raise HTTPException(
163
+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
164
+ detail="Model pipeline is not ready. Try again shortly.",
165
+ )
166
+ return _pipeline
167
+
168
+
169
+ def _get_gradcam() -> GradCAM:
170
+ """Lazy-load GradCAM on first explainability request (histopathology)."""
171
+ global _gradcam
172
+ if _gradcam is None:
173
+ pipeline = _get_pipeline()
174
+ alpha = float(os.getenv("GRADCAM_ALPHA", "0.5"))
175
+ logger.info("Initialising Grad-CAM module (device=%s, alpha=%.2f)...",
176
+ pipeline.device, alpha)
177
+ _gradcam = GradCAM(
178
+ model = pipeline.model,
179
+ device = str(pipeline.device),
180
+ alpha = alpha,
181
+ )
182
+ logger.info("Grad-CAM ready.")
183
+ return _gradcam
184
+
185
+
186
+ def _get_llm() -> LLMExplainer:
187
+ global _llm_explainer
188
+ if _llm_explainer is None:
189
+ use_llm = os.getenv("USE_LLM_MODEL", "false").lower() == "true"
190
+ model_name = os.getenv("LLM_MODEL_NAME", "google/flan-t5-base")
191
+ logger.info(
192
+ "Initialising LLM explainer (use_llm=%s, model=%s)...",
193
+ use_llm, model_name if use_llm else "template-engine",
194
+ )
195
+ _llm_explainer = LLMExplainer(model_name=model_name, use_llm=use_llm)
196
+ logger.info("LLM explainer ready (engine=%s).",
197
+ "flan-t5" if use_llm else "template")
198
+ return _llm_explainer
199
+
200
+
201
+ def _get_chat_engine() -> ChatEngine:
202
+ global _chat_engine
203
+ if _chat_engine is None:
204
+ _chat_engine = ChatEngine()
205
+ logger.info("ChatEngine fallback ready.")
206
+ return _chat_engine
207
+
208
+
209
+ def _get_chat_pipeline() -> "DualModelChatPipeline | None":
210
+ global _chat_pipeline
211
+ if _chat_pipeline is None:
212
+ _chat_pipeline = create_pipeline()
213
+ if _chat_pipeline:
214
+ logger.info("Dual pipeline ready: BioMedLM + Llama 3.2 (Groq)")
215
+ else:
216
+ logger.info(
217
+ "Dual pipeline not available β€” set GROQ_API_KEY + HF_TOKEN "
218
+ "to enable BioMedLM + Llama 3.2. Using ChatEngine fallback."
219
+ )
220
+ return _chat_pipeline
221
+
222
+
223
+ # ── Response schemas ──────────────────────────────────────────────────────────
224
+
225
+ class PredictionResponse(BaseModel):
226
+ prediction: str = Field(..., examples=["malignant"])
227
+ confidence: float = Field(..., ge=0.0, le=1.0, examples=[0.875])
228
+ logits: list[float]= Field(
229
+ ...,
230
+ description="Raw pre-softmax scores [benign, malignant]",
231
+ examples=[[-2.14, 3.87]],
232
+ )
233
+ disclaimer: str = Field(
234
+ default=("Research / educational use only. "
235
+ "Not a standalone diagnostic tool.")
236
+ )
237
+
238
+
239
+ class ExplainResponse(PredictionResponse):
240
+ summary: str = Field(..., description="Plain-language summary of the prediction")
241
+ detail: str = Field(..., description="Deeper explanation with confidence context")
242
+ audience: str = Field(..., description="Target audience the explanation is tailored for")
243
+ engine: str = Field(..., description="'flan-t5' if LLM ran, 'template' if fallback")
244
+
245
+
246
+ class VisualExplainResponse(ExplainResponse):
247
+ overlay_b64: str = Field(..., description="Base64 PNG of Grad-CAM overlay")
248
+ heatmap_b64: str = Field(..., description="Base64 PNG of raw Grad-CAM heatmap")
249
+ spatial_summary: str = Field(..., description="Regions that drove the prediction")
250
+ heatmap_mean: float = Field(..., description="Mean activation value [0,1]")
251
+ heatmap_max: float = Field(..., description="Peak activation value [0,1]")
252
+
253
+
254
+ class HealthResponse(BaseModel):
255
+ status: str
256
+ device: str
257
+ gradcam_loaded: bool
258
+ llm_loaded: bool
259
+ llm_engine: str
260
+ mammogram_mode: str
261
+ chat_llm: str
262
+
263
+
264
+ # ── Helpers ───────────────────────────────────────────────────────────────────
265
+ ACCEPTED_MIME = {"image/jpeg", "image/png", "image/tiff", "image/bmp"}
266
+
267
+
268
+ def _validate_and_open(upload: UploadFile) -> Image.Image:
269
+ if upload.content_type not in ACCEPTED_MIME:
270
+ raise HTTPException(
271
+ status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
272
+ detail=f"Unsupported file type: {upload.content_type}. "
273
+ f"Accepted: {', '.join(ACCEPTED_MIME)}",
274
+ )
275
+ data = upload.file.read()
276
+ try:
277
+ return Image.open(io.BytesIO(data)).convert("RGB")
278
+ except Exception as exc:
279
+ raise HTTPException(
280
+ status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
281
+ detail=f"Could not decode image: {exc}",
282
+ ) from exc
283
+
284
+
285
+ def _serialize(result: dict) -> dict:
286
+ return {
287
+ "prediction": result["prediction"],
288
+ "confidence": result["confidence"],
289
+ "logits": result["logits"].squeeze().tolist(),
290
+ }
291
+
292
+
293
+ def _pil_to_b64(img: Image.Image, fmt: str = "PNG") -> str:
294
+ buf = io.BytesIO()
295
+ img.save(buf, format=fmt)
296
+ return base64.b64encode(buf.getvalue()).decode("utf-8")
297
+
298
+
299
+ def _heatmap_to_pil(heatmap) -> Image.Image:
300
+ import numpy as np
301
+ arr = (heatmap * 255).astype(np.uint8)
302
+ return Image.fromarray(arr, mode="L")
303
+
304
+
305
+ def _ensemble_logits(result: dict) -> list[float]:
306
+ """
307
+ The ensemble returns averaged probabilities, not raw logits. Synthesize
308
+ log-probabilities so the response schema and the /chat context (which
309
+ expects [benign, malignant] scores) stay backward-compatible.
310
+ """
311
+ mal = float(result.get("malignant_probability", result.get("confidence", 0.5)))
312
+ ben = 1.0 - mal
313
+ return [round(math.log(max(ben, 1e-6)), 6),
314
+ round(math.log(max(mal, 1e-6)), 6)]
315
+
316
+
317
+ def _is_ensemble(pipeline) -> bool:
318
+ return isinstance(pipeline, MammogramEnsemble)
319
+
320
+
321
+ class PatientRecord(BaseModel):
322
+ name: str = Field(default="", description="Patient name")
323
+ age: int = Field(default=0, description="Patient age")
324
+ sex: str = Field(default="", description="Male / Female / Other")
325
+ medical_history: str = Field(default="", description="Relevant medical history")
326
+ symptoms: str = Field(default="", description="Current symptoms or concerns")
327
+ previous_scans: str = Field(default="", description="Previous imaging history")
328
+
329
+
330
+ class ChatRequest(BaseModel):
331
+ message: str
332
+ audience: str = Field(default="clinician",
333
+ description="clinician | researcher | patient")
334
+ prediction: str = Field(default="")
335
+ confidence: float = Field(default=0.0, ge=0.0, le=1.0)
336
+ logits: list[float] = Field(default=[0.0, 0.0])
337
+ spatial_summary: str = Field(default="")
338
+ history: list = Field(default_factory=list)
339
+ patient: PatientRecord = Field(default_factory=PatientRecord)
340
+
341
+
342
+ class ChatResponse(BaseModel):
343
+ response: str
344
+ audience: str
345
+
346
+
347
+ # ── Endpoints ─────────────────────────────────────────────────────────────────
348
+
349
+ @app.get("/health", response_model=HealthResponse, tags=["Meta"])
350
+ def health() -> HealthResponse:
351
+ pipeline = _get_pipeline()
352
+ llm_engine = "not_loaded"
353
+ if _llm_explainer is not None:
354
+ llm_engine = "flan-t5" if _llm_explainer.use_llm else "template"
355
+
356
+ mammo_mode = "not_loaded"
357
+ if _mammo_pipeline is not None:
358
+ mammo_mode = "ensemble" if _is_ensemble(_mammo_pipeline) else "single-view"
359
+
360
+ import os
361
+ avail, provider = llm_available()
362
+ model = os.getenv("CHAT_MODEL") or ("openai/gpt-oss-120b" if provider == "groq" else provider)
363
+ chat_llm = f"{provider}/{model}" if avail else f"{provider}/no_api_key"
364
+
365
+ return HealthResponse(
366
+ status = "ok",
367
+ device = str(pipeline.device),
368
+ gradcam_loaded = _gradcam is not None,
369
+ llm_loaded = _llm_explainer is not None,
370
+ llm_engine = llm_engine,
371
+ mammogram_mode = mammo_mode,
372
+ chat_llm = chat_llm,
373
+ )
374
+
375
+
376
+ @app.post("/predict", response_model=PredictionResponse, tags=["Inference"])
377
+ def predict(file: UploadFile = File(...)) -> PredictionResponse:
378
+ pipeline = _get_pipeline()
379
+ image = _validate_and_open(file)
380
+ try:
381
+ result = pipeline.predict(image)
382
+ except Exception as exc:
383
+ logger.exception("Inference error")
384
+ raise HTTPException(500, detail=f"Inference failed: {exc}") from exc
385
+ return PredictionResponse(**_serialize(result))
386
+
387
+
388
+ @app.post("/predict/batch", response_model=List[PredictionResponse], tags=["Inference"])
389
+ def predict_batch(files: List[UploadFile] = File(...)) -> list[PredictionResponse]:
390
+ if len(files) > 16:
391
+ raise HTTPException(400, detail="Batch size exceeds maximum of 16 images.")
392
+ pipeline = _get_pipeline()
393
+ images = [_validate_and_open(f) for f in files]
394
+ try:
395
+ results = pipeline.predict_batch(images)
396
+ except Exception as exc:
397
+ logger.exception("Batch inference error")
398
+ raise HTTPException(500, detail=f"Batch inference failed: {exc}") from exc
399
+ return [PredictionResponse(**_serialize(r)) for r in results]
400
+
401
+
402
+ @app.post("/explain", response_model=ExplainResponse, tags=["Explainability"])
403
+ def explain(
404
+ file: UploadFile = File(...),
405
+ audience: str = Query(default="clinician",
406
+ enum=["clinician", "researcher", "patient"]),
407
+ ) -> ExplainResponse:
408
+ pipeline = _get_pipeline()
409
+ llm = _get_llm()
410
+ image = _validate_and_open(file)
411
+ try:
412
+ prediction = pipeline.predict(image)
413
+ except Exception as exc:
414
+ logger.exception("Inference error in /explain")
415
+ raise HTTPException(500, detail=f"Inference failed: {exc}") from exc
416
+ try:
417
+ report = llm.explain(prediction, audience=audience)
418
+ except Exception as exc:
419
+ logger.exception("LLM explanation error in /explain")
420
+ raise HTTPException(500, detail=f"Explanation generation failed: {exc}") from exc
421
+ return ExplainResponse(
422
+ **_serialize(prediction),
423
+ summary = report["summary"],
424
+ detail = report["detail"],
425
+ audience = report["audience"],
426
+ engine = report["engine"],
427
+ )
428
+
429
+
430
+ @app.post("/explain/visual", response_model=VisualExplainResponse, tags=["Explainability"])
431
+ def explain_visual(
432
+ file: UploadFile = File(...),
433
+ audience: str = Query(default="clinician",
434
+ enum=["clinician", "researcher", "patient"]),
435
+ class_idx: Optional[int] = Query(default=None, ge=0, le=1),
436
+ ) -> VisualExplainResponse:
437
+ llm = _get_llm()
438
+ cam = _get_gradcam()
439
+ image = _validate_and_open(file)
440
+ try:
441
+ cam_result = cam.explain(image, class_idx=class_idx)
442
+ except Exception as exc:
443
+ logger.exception("Grad-CAM error in /explain/visual")
444
+ raise HTTPException(500, detail=f"Grad-CAM failed: {exc}") from exc
445
+ try:
446
+ report = llm.explain_with_gradcam(cam_result, audience=audience)
447
+ except Exception as exc:
448
+ logger.exception("LLM explanation error in /explain/visual")
449
+ raise HTTPException(500, detail=f"Explanation generation failed: {exc}") from exc
450
+
451
+ import numpy as np
452
+ heatmap = cam_result["heatmap"]
453
+ overlay = cam_result["overlay"]
454
+ return VisualExplainResponse(
455
+ **_serialize(cam_result),
456
+ summary = report["summary"],
457
+ detail = report["detail"],
458
+ audience = report["audience"],
459
+ engine = report["engine"],
460
+ overlay_b64 = _pil_to_b64(overlay),
461
+ heatmap_b64 = _pil_to_b64(_heatmap_to_pil(heatmap)),
462
+ spatial_summary = LLMExplainer._summarise_heatmap(heatmap),
463
+ heatmap_mean = round(float(np.mean(heatmap)), 6),
464
+ heatmap_max = round(float(np.max(heatmap)), 6),
465
+ )
466
+
467
+
468
+ @app.post("/chat", response_model=ChatResponse, tags=["Explainability"])
469
+ def chat(request: ChatRequest) -> ChatResponse:
470
+ logits = request.logits if len(request.logits) >= 2 else [0.0, 0.0]
471
+ patient_dict = {}
472
+ if request.patient:
473
+ patient_dict = {
474
+ "name": request.patient.name,
475
+ "age": request.patient.age,
476
+ "sex": request.patient.sex,
477
+ "medical_history": request.patient.medical_history,
478
+ "symptoms": request.patient.symptoms,
479
+ "previous_scans": request.patient.previous_scans,
480
+ }
481
+
482
+ pipeline = _get_chat_pipeline()
483
+ if pipeline:
484
+ try:
485
+ response_text = pipeline.respond(
486
+ message = request.message,
487
+ audience = request.audience,
488
+ prediction = request.prediction,
489
+ confidence = request.confidence,
490
+ benign_logit = logits[0],
491
+ malignant_logit = logits[1],
492
+ spatial_summary = request.spatial_summary,
493
+ history = request.history,
494
+ patient = patient_dict,
495
+ )
496
+ if response_text:
497
+ return ChatResponse(response=response_text, audience=request.audience)
498
+ except Exception as e:
499
+ logger.warning("Dual pipeline error (%s) β€” falling back to ChatEngine.", e)
500
+
501
+ _get_llm()
502
+ engine = _get_chat_engine()
503
+ response_text = engine.respond(
504
+ message = request.message,
505
+ audience = request.audience,
506
+ prediction = request.prediction,
507
+ confidence = request.confidence,
508
+ benign_logit = logits[0],
509
+ malignant_logit = logits[1],
510
+ spatial_summary = request.spatial_summary,
511
+ history = request.history,
512
+ patient = patient_dict,
513
+ )
514
+ return ChatResponse(response=response_text, audience=request.audience)
515
+
516
+
517
+ class LLMChatRequest(BaseModel):
518
+ messages: list = Field(default_factory=list,
519
+ description="[{role: 'user'|'assistant', content: str}, ...]")
520
+ audience: str = Field(default="clinician")
521
+ prediction: str = Field(default="")
522
+ confidence: float = Field(default=0.0, ge=0.0, le=1.0)
523
+ logits: list[float] = Field(default=[0.0, 0.0])
524
+ spatial_summary: str = Field(default="")
525
+ birads: str = Field(default="")
526
+
527
+
528
+ @app.post("/chat/stream", tags=["Explainability"])
529
+ def chat_stream(req: LLMChatRequest):
530
+ """Real LLM assistant with token streaming (Claude or GPT via env config)."""
531
+ context = {
532
+ "prediction": req.prediction,
533
+ "confidence": req.confidence,
534
+ "logits": req.logits,
535
+ "spatial_summary": req.spatial_summary,
536
+ "birads": req.birads,
537
+ } if req.prediction else None
538
+
539
+ def generate():
540
+ try:
541
+ for token in stream_chat(req.messages, context=context, audience=req.audience):
542
+ yield token
543
+ except Exception as e: # noqa: BLE001
544
+ logger.warning("LLM chat stream error: %s", e)
545
+ yield f"\n[Assistant error: {e}]"
546
+
547
+ return StreamingResponse(generate(), media_type="text/plain; charset=utf-8")
548
+
549
+ class MammogramResponse(BaseModel):
550
+ prediction: str
551
+ confidence: float
552
+ logits: list[float]
553
+ birads: str
554
+ modality: str = "mammogram"
555
+ per_model: Optional[list[float]] = Field(
556
+ default=None,
557
+ description="Per-member malignant probabilities (ensemble mode only)",
558
+ )
559
+ n_models: Optional[int] = Field(
560
+ default=None, description="Number of ensemble members averaged",
561
+ )
562
+ disclaimer: str = (
563
+ "AI-assisted mammogram analysis. Research use only. "
564
+ "Not a substitute for radiologist review or clinical diagnosis."
565
+ )
566
+
567
+
568
+ class MammogramExplainResponse(MammogramResponse):
569
+ summary: str
570
+ detail: str
571
+ audience: str
572
+ engine: str
573
+
574
+
575
+ class MammogramVisualResponse(MammogramExplainResponse):
576
+ overlay_b64: str
577
+ heatmap_b64: str
578
+ spatial_summary: str
579
+ heatmap_mean: float
580
+ heatmap_max: float
581
+
582
+
583
+ def _run_mammo_predict(pipeline, image: Image.Image, tta: bool) -> dict:
584
+ """Unified prediction β†’ dict with prediction/confidence/logits/birads/etc."""
585
+ if _is_ensemble(pipeline):
586
+ result = pipeline.predict_tta(image) if tta else pipeline.predict(image)
587
+ return {
588
+ "prediction": result["prediction"],
589
+ "confidence": result["confidence"],
590
+ "logits": _ensemble_logits(result),
591
+ "birads": result["birads"],
592
+ "modality": result.get("modality", "mammogram_ensemble"),
593
+ "per_model": result.get("per_model"),
594
+ "n_models": result.get("n_models"),
595
+ }
596
+ # Legacy single-view pipeline (returns logits tensor)
597
+ result = pipeline.predict(image)
598
+ return {
599
+ "prediction": result["prediction"],
600
+ "confidence": result["confidence"],
601
+ "logits": result["logits"].squeeze().tolist(),
602
+ "birads": result["birads"],
603
+ "modality": result.get("modality", "mammogram"),
604
+ "per_model": None,
605
+ "n_models": None,
606
+ }
607
+
608
+
609
+ @app.post("/mammogram/predict",
610
+ response_model=MammogramResponse,
611
+ tags=["Mammogram"])
612
+ def mammogram_predict(
613
+ file: UploadFile = File(...),
614
+ tta: bool = Query(default=False,
615
+ description="Test-time augmentation (ensemble only) β€” "
616
+ "slower, slightly more accurate"),
617
+ ) -> MammogramResponse:
618
+ """
619
+ Classify a mammogram using the 3-model EfficientNet-B4 ensemble.
620
+ Returns prediction, confidence, BI-RADS suggestion, and per-member probs.
621
+ """
622
+ pipeline = _get_mammo_pipeline()
623
+ image = _validate_and_open(file)
624
+ try:
625
+ result = _run_mammo_predict(pipeline, image, tta)
626
+ except Exception as exc:
627
+ logger.exception("Mammogram inference error")
628
+ raise HTTPException(500, detail=f"Mammogram inference failed: {exc}") from exc
629
+ return MammogramResponse(**result)
630
+
631
+
632
+ @app.post("/mammogram/explain",
633
+ response_model=MammogramExplainResponse,
634
+ tags=["Mammogram"])
635
+ def mammogram_explain(
636
+ file: UploadFile = File(...),
637
+ audience: str = Query(default="clinician",
638
+ enum=["clinician", "researcher", "patient"]),
639
+ tta: bool = Query(default=False),
640
+ ) -> MammogramExplainResponse:
641
+ """Classify a mammogram (ensemble) and generate an audience-specific explanation."""
642
+ pipeline = _get_mammo_pipeline()
643
+ llm = _get_llm()
644
+ image = _validate_and_open(file)
645
+
646
+ try:
647
+ result = _run_mammo_predict(pipeline, image, tta)
648
+ except Exception as exc:
649
+ raise HTTPException(500, detail=f"Mammogram inference failed: {exc}") from exc
650
+
651
+ explain_input = {
652
+ "prediction": result["prediction"],
653
+ "confidence": result["confidence"],
654
+ "logits": torch.tensor([result["logits"]]),
655
+ "birads": result["birads"],
656
+ }
657
+ try:
658
+ report = llm.explain(explain_input, audience=audience, modality="mammogram")
659
+ except Exception as exc:
660
+ raise HTTPException(500, detail=f"Explanation failed: {exc}") from exc
661
+
662
+ return MammogramExplainResponse(
663
+ **result,
664
+ summary = report["summary"],
665
+ detail = report["detail"],
666
+ audience = report["audience"],
667
+ engine = report["engine"],
668
+ )
669
+
670
+
671
+ @app.post("/mammogram/visual",
672
+ response_model=MammogramVisualResponse,
673
+ tags=["Mammogram"])
674
+ def mammogram_visual(
675
+ file: UploadFile = File(...),
676
+ audience: str = Query(default="clinician",
677
+ enum=["clinician", "researcher", "patient"]),
678
+ class_idx: Optional[int] = Query(default=None, ge=0, le=1),
679
+ ) -> MammogramVisualResponse:
680
+ """
681
+ Full mammogram analysis: ensemble prediction + Grad-CAM + LLM explanation.
682
+
683
+ Grad-CAM is inherently a single-model technique, so the heatmap is computed
684
+ on the strongest ensemble member (the most representative saliency map),
685
+ while the prediction label/confidence uses the full ensemble average.
686
+ """
687
+ import numpy as np
688
+
689
+ pipeline = _get_mammo_pipeline()
690
+ llm = _get_llm()
691
+ image = _validate_and_open(file)
692
+
693
+ # Authoritative label/confidence from the full ensemble
694
+ try:
695
+ pred = _run_mammo_predict(pipeline, image, tta=False)
696
+ except Exception as exc:
697
+ raise HTTPException(500, detail=f"Mammogram inference failed: {exc}") from exc
698
+
699
+ # Grad-CAM on the representative single model (EfficientNet-specific)
700
+ cam_model = pipeline.cam_model if _is_ensemble(pipeline) else pipeline.model
701
+ try:
702
+ alpha = float(os.getenv("GRADCAM_ALPHA", "0.5"))
703
+ mammo_cam = MammogramGradCAM(model=cam_model, device=str(pipeline.device), alpha=alpha)
704
+ cam_result = mammo_cam.explain(image, class_idx=class_idx)
705
+ except Exception as exc:
706
+ logger.exception("Mammogram Grad-CAM error")
707
+ raise HTTPException(
708
+ 500,
709
+ detail=(f"Grad-CAM failed: {exc}. The /mammogram/predict endpoint "
710
+ f"still works without the heatmap."),
711
+ ) from exc
712
+
713
+ # Make the LLM explanation reflect the ENSEMBLE verdict (authoritative),
714
+ # while the heatmap stays the representative member's saliency map.
715
+ cam_result["prediction"] = pred["prediction"]
716
+ cam_result["confidence"] = pred["confidence"]
717
+ cam_result["logits"] = torch.tensor([pred["logits"]])
718
+ cam_result["birads"] = pred["birads"]
719
+
720
+ try:
721
+ report = llm.explain_with_gradcam(cam_result, audience=audience, modality="mammogram")
722
+ except Exception as exc:
723
+ raise HTTPException(500, detail=f"Explanation failed: {exc}") from exc
724
+
725
+ heatmap = cam_result["heatmap"]
726
+ overlay = cam_result["overlay"]
727
+ return MammogramVisualResponse(
728
+ prediction = pred["prediction"],
729
+ confidence = pred["confidence"],
730
+ logits = pred["logits"],
731
+ birads = pred["birads"],
732
+ modality = pred["modality"],
733
+ per_model = pred["per_model"],
734
+ n_models = pred["n_models"],
735
+ summary = report["summary"],
736
+ detail = report["detail"],
737
+ audience = report["audience"],
738
+ engine = report["engine"],
739
+ overlay_b64 = _pil_to_b64(overlay),
740
+ heatmap_b64 = _pil_to_b64(_heatmap_to_pil(heatmap)),
741
+ spatial_summary = LLMExplainer._summarise_heatmap(heatmap),
742
+ heatmap_mean = round(float(np.mean(heatmap)), 6),
743
+ heatmap_max = round(float(np.max(heatmap)), 6),
744
+ )
explainability/chat_pipeline.py ADDED
@@ -0,0 +1,568 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ explainability/chat_pipeline.py
3
+ ────────────────────────────────
4
+ Dual-model conversational pipeline for the MedAI chat assistant.
5
+
6
+ Architecture
7
+ ────────────
8
+ Stage 1 β€” BioMedLM (HuggingFace Inference API, free)
9
+ ↓ Enriches the scan context with clinical knowledge:
10
+ medical terminology, risk factors, differential diagnosis,
11
+ clinical recommendations, and pathology context.
12
+
13
+ Stage 2 β€” Llama 3.2 (Groq API, free tier)
14
+ ↓ Takes BioMedLM's enrichment + patient record + scan results
15
+ and produces a warm, natural, human-like conversation response
16
+ tailored to the selected audience.
17
+
18
+ Why two models
19
+ ──────────────
20
+ BioMedLM knows medicine deeply β€” it was trained on PubMed abstracts
21
+ and biomedical literature. It understands BRCA1, BI-RADS categories,
22
+ nuclear atypia, ductal carcinoma, and thousands of clinical concepts.
23
+
24
+ Llama 3.2 is a world-class conversational model β€” it produces warm,
25
+ empathetic, natural language. It can switch between clinical colleague
26
+ tone, researcher-level technical depth, and patient-friendly plain English.
27
+
28
+ Together: BioMedLM provides the medical substance.
29
+ Llama 3.2 provides the human voice.
30
+
31
+ Environment variables required
32
+ ──────────────────────────────
33
+ GROQ_API_KEY β€” from console.groq.com (free, no card)
34
+ HF_TOKEN β€” from huggingface.co/settings/tokens (free)
35
+
36
+ Fallback behaviour
37
+ ──────────────────
38
+ If BioMedLM fails (cold start, rate limit) β†’ Llama 3.2 runs alone.
39
+ If Groq fails (rate limit) β†’ deterministic ChatEngine.
40
+ If both fail β†’ ChatEngine always responds.
41
+ The system never returns an empty response.
42
+
43
+ Install
44
+ ───────
45
+ pip install groq requests
46
+ """
47
+
48
+ from __future__ import annotations
49
+
50
+ import logging
51
+ import os
52
+ import time
53
+ from typing import Optional
54
+
55
+ import requests
56
+
57
+ logger = logging.getLogger(__name__)
58
+
59
+
60
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
61
+ # β•‘ STAGE 1 β€” BioMedLM ENRICHER (HuggingFace Inference API) β•‘
62
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
63
+
64
+ class BioMedEnricher:
65
+ """
66
+ Calls BioMedLM via the HuggingFace free Inference API to extract
67
+ clinically relevant context from the scan and patient data.
68
+
69
+ BioMedLM (stanford-crfm/BioMedLM) is a 2.7B GPT-style model trained
70
+ on PubMed abstracts. It generates medical text continuations, which
71
+ we use to enrich the context before Llama 3.2 formulates the response.
72
+
73
+ Parameters
74
+ ----------
75
+ hf_token : str
76
+ HuggingFace access token. Free at huggingface.co/settings/tokens.
77
+ timeout : int
78
+ HTTP timeout in seconds. Default 20.
79
+ max_retries : int
80
+ Retries if model is loading (cold start returns 503). Default 2.
81
+ """
82
+
83
+ MODEL_URL = (
84
+ "https://router.huggingface.co/hf-inference/models/stanford-crfm/BioMedLM"
85
+ )
86
+
87
+ def __init__(
88
+ self,
89
+ hf_token: str,
90
+ timeout: int = 20,
91
+ max_retries: int = 2,
92
+ ) -> None:
93
+ self.headers = {"Authorization": f"Bearer {hf_token}"}
94
+ self.timeout = timeout
95
+ self.max_retries = max_retries
96
+
97
+ def enrich(
98
+ self,
99
+ prediction: str,
100
+ confidence: float,
101
+ benign_logit: float,
102
+ malignant_logit: float,
103
+ spatial_summary: str,
104
+ patient: dict,
105
+ question: str,
106
+ ) -> str:
107
+ """
108
+ Generate a clinical enrichment passage using BioMedLM.
109
+
110
+ Builds a structured medical prompt, sends it to the HuggingFace
111
+ Inference API, and returns the generated clinical context.
112
+
113
+ Returns empty string if the API call fails β€” the pipeline
114
+ continues without enrichment in that case.
115
+ """
116
+ prompt = self._build_prompt(
117
+ prediction, confidence, benign_logit, malignant_logit,
118
+ spatial_summary, patient, question
119
+ )
120
+
121
+ for attempt in range(self.max_retries + 1):
122
+ try:
123
+ resp = requests.post(
124
+ self.MODEL_URL,
125
+ headers = self.headers,
126
+ json = {
127
+ "inputs": prompt,
128
+ "parameters": {
129
+ "max_new_tokens": 180,
130
+ "temperature": 0.3, # low temp for factual output
131
+ "repetition_penalty": 1.3,
132
+ "return_full_text": False, # only return generated part
133
+ },
134
+ },
135
+ timeout = self.timeout,
136
+ )
137
+
138
+ # Model loading (cold start) β€” wait and retry
139
+ if resp.status_code == 503:
140
+ data = resp.json()
141
+ wait = data.get("estimated_time", 8)
142
+ logger.info(
143
+ "BioMedLM loading (cold start) β€” waiting %.0fs (attempt %d/%d)",
144
+ wait, attempt + 1, self.max_retries + 1
145
+ )
146
+ time.sleep(min(wait, 15))
147
+ continue
148
+
149
+ if resp.status_code != 200:
150
+ logger.warning(
151
+ "BioMedLM API error %d: %s", resp.status_code, resp.text[:200]
152
+ )
153
+ return ""
154
+
155
+ result = resp.json()
156
+
157
+ # HF returns list of dicts or a single dict
158
+ if isinstance(result, list) and result:
159
+ generated = result[0].get("generated_text", "")
160
+ elif isinstance(result, dict):
161
+ generated = result.get("generated_text", "")
162
+ else:
163
+ return ""
164
+
165
+ # Clean up β€” remove the prompt prefix if model echoes it
166
+ if generated.startswith(prompt):
167
+ generated = generated[len(prompt):]
168
+
169
+ enrichment = generated.strip()
170
+ if enrichment:
171
+ logger.info("BioMedLM enrichment: %d words", len(enrichment.split()))
172
+ return enrichment
173
+
174
+ except requests.exceptions.Timeout:
175
+ logger.warning("BioMedLM request timed out (attempt %d)", attempt + 1)
176
+ except Exception as e:
177
+ logger.warning("BioMedLM error: %s", e)
178
+ return ""
179
+
180
+ return ""
181
+
182
+ def _build_prompt(
183
+ self,
184
+ prediction: str,
185
+ confidence: float,
186
+ benign_logit: float,
187
+ malignant_logit: float,
188
+ spatial_summary: str,
189
+ patient: dict,
190
+ question: str,
191
+ ) -> str:
192
+ """Build a medical text prompt for BioMedLM completion."""
193
+ pct = f"{confidence:.1%}"
194
+ margin = abs(malignant_logit - benign_logit)
195
+ name = patient.get("name", "")
196
+ age = patient.get("age", 0)
197
+ hist = patient.get("medical_history", "")
198
+ symp = patient.get("symptoms", "")
199
+
200
+ patient_ctx = ""
201
+ if age:
202
+ patient_ctx += f"Patient age: {age}."
203
+ if hist:
204
+ patient_ctx += f" Medical history: {hist}."
205
+ if symp:
206
+ patient_ctx += f" Current symptoms: {symp}."
207
+
208
+ return (
209
+ f"Histopathology AI result: {prediction.upper()} ({pct} confidence). "
210
+ f"Logit margin: {margin:.3f}. "
211
+ f"Grad-CAM spatial findings: {spatial_summary or 'not available'}. "
212
+ f"{patient_ctx} "
213
+ f"Clinical question: {question}. "
214
+ f"Medical assessment and clinical implications:"
215
+ )
216
+
217
+
218
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
219
+ # β•‘ STAGE 2 β€” LLAMA 3.2 via GROQ (conversational response) β•‘
220
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
221
+
222
+ class GroqChat:
223
+ """
224
+ Calls Llama 3.2 via the Groq API to generate a natural, human-like
225
+ conversational response.
226
+
227
+ Groq uses custom LPU chips that are ~5x faster than standard GPU inference.
228
+ Responses typically arrive in 0.2–0.5 seconds on the free tier.
229
+
230
+ Free tier limits (as of 2025):
231
+ - Llama 3.2 3B: 30 requests/min, 500 requests/day
232
+ - Llama 3.1 8B: 30 requests/min, 6,000 tokens/min
233
+
234
+ Parameters
235
+ ----------
236
+ api_key : str
237
+ Groq API key. Free at console.groq.com (no card required).
238
+ model : str
239
+ Groq model identifier. Default: llama-3.2-3b-preview.
240
+ """
241
+
242
+ # Audience-specific system prompts β€” each produces a different voice
243
+ SYSTEM_PROMPTS = {
244
+ "clinician": (
245
+ "You are a specialist AI assistant for the MedAI breast cancer analysis platform, "
246
+ "speaking with a consultant radiologist or clinical specialist. "
247
+ "Be precise, collegial, and technically accurate. Use clinical terminology naturally. "
248
+ "Reference BI-RADS categories, biopsy recommendations, logit margins, and sensitivity "
249
+ "metrics where relevant. Be direct β€” clinicians appreciate concise, information-dense responses. "
250
+ "Always acknowledge the AI-assisted nature of the analysis and the need for clinical confirmation."
251
+ ),
252
+ "researcher": (
253
+ "You are a specialist AI assistant for the MedAI breast cancer analysis platform, "
254
+ "speaking with an ML researcher. Be technical and precise. "
255
+ "Discuss logit scores, softmax probabilities, decision boundaries, model architecture "
256
+ "(DenseNet-121, 7.22M params), training methodology (OneCycleLR max_lr=3e-3, Mixup Ξ±=0.4, "
257
+ "StainJitter HED, label_smoothing=0.1, pos_weight=1.469), and calibration freely. "
258
+ "Reference the PCam dataset (220,025 deduplicated patches), test accuracy 88.0%, "
259
+ "test sensitivity 87.5% (epoch 13 checkpoint, selected by val_sensitivity). "
260
+ "Be rigorous and engage at a peer level."
261
+ ),
262
+ "patient": (
263
+ "You are a warm, empathetic AI assistant for the MedAI breast cancer analysis platform, "
264
+ "speaking with a patient who has no medical background. "
265
+ "Use plain English β€” no jargon, no acronyms, no intimidating terminology. "
266
+ "Be genuinely human and warm. Acknowledge their emotions. Use simple analogies. "
267
+ "If the patient seems worried, be reassuring without being dismissive. "
268
+ "If they ask about something serious, be honest but kind. "
269
+ "Always remind them that their doctor makes all final decisions β€” you are a support tool, not a doctor. "
270
+ "Address them by name if you know it. Keep sentences short and clear."
271
+ ),
272
+ }
273
+
274
+ def __init__(
275
+ self,
276
+ api_key: str,
277
+ model: str = "llama-3.1-8b-instant",
278
+ ) -> None:
279
+ try:
280
+ from groq import Groq
281
+ self._client = Groq(api_key=api_key)
282
+ self._model = model
283
+ self._ok = True
284
+ logger.info("Groq client initialised (model=%s)", model)
285
+ except ImportError:
286
+ logger.warning(
287
+ "groq package not installed. Run: pip install groq"
288
+ )
289
+ self._ok = False
290
+
291
+ def generate(
292
+ self,
293
+ audience: str,
294
+ user_message: str,
295
+ scan_context: str,
296
+ patient_context: str,
297
+ biomedlm_context: str,
298
+ history: list,
299
+ ) -> str:
300
+ """
301
+ Generate a conversational response using Llama 3.2 via Groq.
302
+
303
+ Parameters
304
+ ----------
305
+ audience : clinician | researcher | patient
306
+ user_message : the user's question
307
+ scan_context : structured scan result summary
308
+ patient_context : patient name, age, history, symptoms
309
+ biomedlm_context : clinical enrichment from BioMedLM (may be empty)
310
+ history : list of prior {"role", "content"} dicts
311
+
312
+ Returns
313
+ -------
314
+ str β€” Llama 3.2 response, or empty string if Groq call fails.
315
+ """
316
+ if not self._ok:
317
+ return ""
318
+
319
+ system = self.SYSTEM_PROMPTS.get(audience, self.SYSTEM_PROMPTS["clinician"])
320
+
321
+ # Build the context block injected into the system prompt
322
+ context_block = f"\n\nCURRENT SCAN CONTEXT:\n{scan_context}"
323
+ if patient_context:
324
+ context_block += f"\n\nPATIENT INFORMATION:\n{patient_context}"
325
+ if biomedlm_context:
326
+ context_block += (
327
+ f"\n\nCLINICAL ENRICHMENT (from BioMedLM medical knowledge model):\n"
328
+ f"{biomedlm_context}\n"
329
+ f"Use the above medical context to inform your response where relevant."
330
+ )
331
+ context_block += (
332
+ "\n\nIMPORTANT: Keep your response focused and conversational. "
333
+ "2-4 sentences for simple questions, more for complex ones. "
334
+ "Be specific β€” reference the actual scan numbers, not generic statements. "
335
+ "Sound like a knowledgeable colleague, not a textbook."
336
+ )
337
+
338
+ system_with_context = system + context_block
339
+
340
+ # Build message history (last 6 turns for context window efficiency)
341
+ messages = []
342
+ for turn in (history or [])[-6:]:
343
+ if isinstance(turn, dict) and "role" in turn and "content" in turn:
344
+ messages.append({
345
+ "role": turn["role"],
346
+ "content": turn["content"],
347
+ })
348
+ messages.append({"role": "user", "content": user_message})
349
+
350
+ try:
351
+ response = self._client.chat.completions.create(
352
+ model = self._model,
353
+ messages = [{"role": "system", "content": system_with_context}] + messages,
354
+ max_tokens = 512,
355
+ temperature = 0.75,
356
+ top_p = 0.9,
357
+ )
358
+ text = response.choices[0].message.content.strip()
359
+ logger.info(
360
+ "Groq response: %d words (model=%s)", len(text.split()), self._model
361
+ )
362
+ return text
363
+
364
+ except Exception as e:
365
+ logger.warning("Groq API error: %s", e)
366
+ return ""
367
+
368
+
369
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
370
+ # β•‘ DUAL-MODEL PIPELINE ORCHESTRATOR β•‘
371
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
372
+
373
+ class DualModelChatPipeline:
374
+ """
375
+ Orchestrates BioMedLM + Llama 3.2 for human-quality medical chat.
376
+
377
+ Stage 1: BioMedLM enriches the context with clinical knowledge.
378
+ Stage 2: Llama 3.2 produces a natural, audience-specific response.
379
+
380
+ Graceful degradation:
381
+ - BioMedLM failure β†’ Llama 3.2 runs without enrichment (still good)
382
+ - Groq failure β†’ falls back to deterministic ChatEngine
383
+ - Both fail β†’ ChatEngine always produces a response
384
+
385
+ Parameters
386
+ ----------
387
+ groq_key : str
388
+ Groq API key (GROQ_API_KEY env var). Free at console.groq.com.
389
+ hf_token : str
390
+ HuggingFace token (HF_TOKEN env var). Free at huggingface.co.
391
+ groq_model : str
392
+ Groq model to use. Default: llama-3.2-3b-preview.
393
+ """
394
+
395
+ def __init__(
396
+ self,
397
+ groq_key: str,
398
+ hf_token: str,
399
+ groq_model: str = "llama-3.1-8b-instant",
400
+ ) -> None:
401
+ self.biomedlm = BioMedEnricher(hf_token=hf_token)
402
+ self.llama = GroqChat(api_key=groq_key, model=groq_model)
403
+ logger.info(
404
+ "DualModelChatPipeline ready (BioMedLM + %s via Groq)", groq_model
405
+ )
406
+
407
+ def respond(
408
+ self,
409
+ message: str,
410
+ audience: str,
411
+ prediction: str,
412
+ confidence: float,
413
+ benign_logit: float,
414
+ malignant_logit: float,
415
+ spatial_summary: str = "",
416
+ history: list = None,
417
+ patient: dict = None,
418
+ ) -> str:
419
+ """
420
+ Generate a human-like chat response using the dual-model pipeline.
421
+
422
+ Parameters
423
+ ----------
424
+ message : user's question
425
+ audience : clinician | researcher | patient
426
+ prediction : benign | malignant
427
+ confidence : float [0, 1]
428
+ benign_logit : raw logit for benign class
429
+ malignant_logit : raw logit for malignant class
430
+ spatial_summary : Grad-CAM text description
431
+ history : conversation history [{"role", "content"}]
432
+ patient : patient record dict
433
+
434
+ Returns
435
+ -------
436
+ str β€” response text, or empty string if both models fail.
437
+ """
438
+ patient = patient or {}
439
+ history = history or []
440
+ margin = abs(malignant_logit - benign_logit)
441
+ pct = f"{confidence:.1%}"
442
+
443
+ # ── Build structured scan context ─────────────────────────────────────
444
+ scan_context = (
445
+ f"Prediction: {prediction.upper()}\n"
446
+ f"Confidence: {pct}\n"
447
+ f"Logits: benign={benign_logit:.4f}, malignant={malignant_logit:.4f}\n"
448
+ f"Decision margin: {margin:.4f} "
449
+ f"({'clear separation' if margin > 1.5 else 'near boundary β€” borderline'})\n"
450
+ f"Grad-CAM: {spatial_summary or 'not available'}\n"
451
+ f"Model: DenseNet-121 | Accuracy: 88.0% | Sensitivity: 87.5%"
452
+ )
453
+
454
+ # ── Build patient context ─────────────────────────────────────────────
455
+ patient_context = self._build_patient_context(patient)
456
+
457
+ # ── Stage 1: BioMedLM enrichment (best-effort) ───────────────────────
458
+ logger.info("Stage 1: BioMedLM enrichment...")
459
+ biomedlm_context = ""
460
+ try:
461
+ biomedlm_context = self.biomedlm.enrich(
462
+ prediction = prediction,
463
+ confidence = confidence,
464
+ benign_logit = benign_logit,
465
+ malignant_logit = malignant_logit,
466
+ spatial_summary = spatial_summary,
467
+ patient = patient,
468
+ question = message,
469
+ )
470
+ except Exception as e:
471
+ logger.warning("BioMedLM enrichment failed: %s β€” continuing without it.", e)
472
+
473
+ if biomedlm_context:
474
+ logger.info("Stage 1 complete: BioMedLM provided %d words of enrichment.",
475
+ len(biomedlm_context.split()))
476
+ else:
477
+ logger.info("Stage 1: No BioMedLM enrichment β€” Llama 3.2 will run standalone.")
478
+
479
+ # ── Stage 2: Llama 3.2 via Groq ──────────────────────────────────────
480
+ logger.info("Stage 2: Llama 3.2 via Groq (audience=%s)...", audience)
481
+ response = self.llama.generate(
482
+ audience = audience,
483
+ user_message = message,
484
+ scan_context = scan_context,
485
+ patient_context = patient_context,
486
+ biomedlm_context = biomedlm_context,
487
+ history = history,
488
+ )
489
+
490
+ if response:
491
+ logger.info("Stage 2 complete: %d word response generated.", len(response.split()))
492
+ return response
493
+
494
+ # ── Both failed ───────────────────────────────────────────────────────
495
+ logger.warning("Both models failed β€” ChatEngine fallback will handle this.")
496
+ return ""
497
+
498
+ @staticmethod
499
+ def _build_patient_context(patient: dict) -> str:
500
+ """Format patient record into a clean context string."""
501
+ parts = []
502
+ if patient.get("name"):
503
+ parts.append(f"Name: {patient['name']}")
504
+ if patient.get("age"):
505
+ parts.append(f"Age: {patient['age']}")
506
+ if patient.get("sex"):
507
+ parts.append(f"Sex: {patient['sex']}")
508
+ if patient.get("medical_history"):
509
+ parts.append(f"Medical history: {patient['medical_history']}")
510
+ if patient.get("symptoms"):
511
+ parts.append(f"Current symptoms: {patient['symptoms']}")
512
+ if patient.get("previous_scans"):
513
+ parts.append(f"Previous scans: {patient['previous_scans']}")
514
+ return "\n".join(parts)
515
+
516
+
517
+ # ── Factory function ──────────────────────────────────────────────────────────
518
+
519
+ def create_pipeline(
520
+ groq_key: Optional[str] = None,
521
+ hf_token: Optional[str] = None,
522
+ groq_model: str = "llama-3.1-8b-instant",
523
+ ) -> Optional[DualModelChatPipeline]:
524
+ """
525
+ Create a DualModelChatPipeline if both API keys are available.
526
+
527
+ Reads from environment variables if keys are not passed directly.
528
+ Returns None if either key is missing β€” the caller should fall back
529
+ to the deterministic ChatEngine.
530
+
531
+ Usage
532
+ -----
533
+ # Set env vars before starting the server:
534
+ export GROQ_API_KEY="gsk_..."
535
+ export HF_TOKEN="hf_..."
536
+
537
+ # In Python:
538
+ pipeline = create_pipeline()
539
+ if pipeline:
540
+ response = pipeline.respond(...)
541
+ else:
542
+ response = chat_engine.respond(...)
543
+ """
544
+ groq_key = groq_key or os.getenv("GROQ_API_KEY", "")
545
+ hf_token = hf_token or os.getenv("HF_TOKEN", "")
546
+
547
+ if not groq_key:
548
+ logger.info(
549
+ "GROQ_API_KEY not set β€” dual pipeline disabled. "
550
+ "Get a free key at console.groq.com"
551
+ )
552
+ return None
553
+
554
+ if not hf_token:
555
+ logger.info(
556
+ "HF_TOKEN not set β€” BioMedLM enrichment disabled. "
557
+ "Llama 3.2 will run without medical enrichment. "
558
+ "Get a free token at huggingface.co/settings/tokens"
559
+ )
560
+ # Can still run with just Groq β€” BioMedEnricher will return ""
561
+ # if called without a valid token
562
+ hf_token = ""
563
+
564
+ return DualModelChatPipeline(
565
+ groq_key = groq_key,
566
+ hf_token = hf_token,
567
+ groq_model = groq_model,
568
+ )
explainability/gradcam.py ADDED
@@ -0,0 +1,340 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ explainability/gradcam.py
3
+ ─────────────────────────
4
+ Gradient-weighted Class Activation Mapping (Grad-CAM) for the
5
+ BreastCancerClassifier.
6
+
7
+ What it does
8
+ ────────────
9
+ Grad-CAM answers the question:
10
+ "Which regions of this histopathology image most influenced
11
+ the model's prediction?"
12
+
13
+ It hooks into the last DenseNet feature layer (denseblock4 β†’ norm5),
14
+ computes gradients of the predicted class score with respect to those
15
+ spatial feature maps, then produces a colour heatmap the same size as
16
+ the input image showing the most diagnostically relevant regions.
17
+
18
+ How it connects to the existing architecture
19
+ ────────────────────────────────────────────
20
+ model/model.py β†’ GradCAM wraps BreastCancerClassifier
21
+ Uses model.features (DenseNet backbone)
22
+ Uses model.classifier (head)
23
+ Uses model.pool (global avg pool)
24
+ Does NOT modify any of them
25
+
26
+ model/inference.py β†’ GradCAM.explain() accepts the same image
27
+ formats as BreastCancerInferencePipeline.predict()
28
+ Returns logits compatible with inference output
29
+
30
+ utils/preprocessing β†’ GradCAM uses ImagePreprocessor internally
31
+
32
+ Usage
33
+ ─────
34
+ import sys
35
+ sys.path.insert(0, "/path/to/medical-ai")
36
+
37
+ from model.model import BreastCancerClassifier
38
+ from explainability import GradCAM
39
+
40
+ model = BreastCancerClassifier(pretrained=False)
41
+ # model.load_state_dict(torch.load("model/weights.pth")["state_dict"])
42
+
43
+ cam = GradCAM(model)
44
+ result = cam.explain("slide.png")
45
+
46
+ # result = {
47
+ # "prediction" : "malignant",
48
+ # "confidence" : 0.934,
49
+ # "logits" : tensor([[-2.14, 3.87]]),
50
+ # "heatmap" : np.ndarray shape (224, 224) values [0, 1]
51
+ # "overlay" : PIL.Image heatmap blended on original image
52
+ # "cam_raw" : np.ndarray unresized raw cam before upscale
53
+ # }
54
+
55
+ result["overlay"].save("gradcam_output.png")
56
+ """
57
+
58
+ from __future__ import annotations
59
+
60
+ import sys
61
+ from pathlib import Path
62
+ from typing import Optional, Union
63
+
64
+ import numpy as np
65
+ import torch
66
+ import torch.nn as nn
67
+ from PIL import Image
68
+
69
+ # Allow running from any working directory
70
+ ROOT = Path(__file__).resolve().parents[1]
71
+ sys.path.insert(0, str(ROOT))
72
+
73
+ from utils.preprocessing import ImagePreprocessor
74
+
75
+ # Label map β€” must match model/inference.py
76
+ LABEL_MAP = {0: "benign", 1: "malignant"}
77
+
78
+ # Colour map for heatmap overlay (BGR β†’ RGB matplotlib-style jet)
79
+ # Pre-computed jet colourmap: value in [0,1] β†’ (R, G, B) in [0, 255]
80
+ def _jet_colormap(value: float) -> tuple[int, int, int]:
81
+ """Map a scalar in [0,1] to an RGB colour using the jet colourmap."""
82
+ v = float(np.clip(value, 0.0, 1.0))
83
+ r = int(np.clip(1.5 - abs(4 * v - 3), 0, 1) * 255)
84
+ g = int(np.clip(1.5 - abs(4 * v - 2), 0, 1) * 255)
85
+ b = int(np.clip(1.5 - abs(4 * v - 1), 0, 1) * 255)
86
+ return r, g, b
87
+
88
+
89
+ def _apply_jet_colormap(heatmap: np.ndarray) -> np.ndarray:
90
+ """
91
+ Convert a (H, W) float array in [0, 1] to a (H, W, 3) uint8 RGB array
92
+ using the jet colourmap. Avoids matplotlib dependency.
93
+ """
94
+ h, w = heatmap.shape
95
+ rgb = np.zeros((h, w, 3), dtype=np.uint8)
96
+ for i in range(h):
97
+ for j in range(w):
98
+ rgb[i, j] = _jet_colormap(heatmap[i, j])
99
+ return rgb
100
+
101
+
102
+ class GradCAM:
103
+ """
104
+ Grad-CAM for BreastCancerClassifier.
105
+
106
+ Hooks into the final DenseNet feature layer (norm5 β€” the batch norm
107
+ after denseblock4) to capture spatial activations and their gradients.
108
+ This layer produces (B, 1024, 7, 7) feature maps β€” the same tensor
109
+ that model.get_feature_maps() returns, now with gradient tracking.
110
+
111
+ Parameters
112
+ ----------
113
+ model : BreastCancerClassifier
114
+ The model to explain. Must be the same BreastCancerClassifier
115
+ from model/model.py β€” no modifications needed.
116
+ device : str | None
117
+ "cuda", "mps", or "cpu". Auto-detected if None.
118
+ target_layer : str
119
+ Name of the layer in model.features to hook.
120
+ Default "norm5" = final BatchNorm after denseblock4.
121
+ This is the deepest spatial layer before global pooling.
122
+ alpha : float
123
+ Blend weight for the overlay image. 0.0 = original only,
124
+ 1.0 = heatmap only. Default 0.5.
125
+ """
126
+
127
+ def __init__(
128
+ self,
129
+ model: "BreastCancerClassifier",
130
+ device: Optional[str] = None,
131
+ target_layer: str = "norm5",
132
+ alpha: float = 0.5,
133
+ ) -> None:
134
+ self.device = self._resolve_device(device)
135
+ self.alpha = alpha
136
+ self.preprocessor = ImagePreprocessor()
137
+
138
+ # Set model to eval β€” Grad-CAM still needs gradients but not
139
+ # dropout randomness. eval() disables dropout, keeps grad flow.
140
+ self.model = model.eval().to(self.device)
141
+
142
+ # ── Hook storage ──────────────────────────────────────────────────────
143
+ # These are filled by the forward and backward hooks on each call.
144
+ self._activations: Optional[torch.Tensor] = None
145
+ self._gradients: Optional[torch.Tensor] = None
146
+
147
+ # ── Register hooks on the target layer ───────────────────────────────
148
+ # target_layer is a named layer inside model.features (DenseNet backbone)
149
+ # "norm5" is the final BatchNorm, output shape: (B, 1024, 7, 7)
150
+ target = dict(model.features.named_modules())[target_layer]
151
+
152
+ # Forward hook: fires after the layer's forward() β€” captures activations
153
+ target.register_forward_hook(self._save_activations)
154
+
155
+ # Backward hook: fires after gradients flow back β€” captures gradients
156
+ target.register_full_backward_hook(self._save_gradients)
157
+
158
+ # ── Hook callbacks ────────────────────────────────────────────────────────
159
+
160
+ def _save_activations(self, module, input, output) -> None:
161
+ """Called automatically after the target layer's forward pass."""
162
+ # Detach from graph β€” we only need the values, not to backprop through
163
+ # the stored activations themselves
164
+ self._activations = output.detach()
165
+
166
+ def _save_gradients(self, module, grad_input, grad_output) -> None:
167
+ """Called automatically during backward pass through the target layer."""
168
+ # grad_output[0] = gradients w.r.t. the layer's output tensor
169
+ self._gradients = grad_output[0].detach()
170
+
171
+ # ── Main API ──────────────────────────────────────────────────────────────
172
+
173
+ def explain(
174
+ self,
175
+ image: Union[str, Path, "Image.Image", np.ndarray, torch.Tensor],
176
+ class_idx: Optional[int] = None,
177
+ ) -> dict:
178
+ """
179
+ Run Grad-CAM on a single histopathology image.
180
+
181
+ Parameters
182
+ ----------
183
+ image : any format accepted by ImagePreprocessor
184
+ The histopathology patch or slide to explain.
185
+ class_idx : int | None
186
+ Which class to generate the heatmap for.
187
+ None (default) = use the predicted class (argmax).
188
+ 0 = benign, 1 = malignant.
189
+
190
+ Returns
191
+ -------
192
+ dict
193
+ {
194
+ "prediction" : str β€” "benign" | "malignant"
195
+ "confidence" : float β€” probability of predicted class
196
+ "logits" : Tensor[1,2] β€” raw pre-softmax scores
197
+ "heatmap" : ndarray β€” (224,224) float in [0,1]
198
+ "overlay" : PIL.Image β€” heatmap blended onto image
199
+ "cam_raw" : ndarray β€” (7,7) unresized raw CAM
200
+ }
201
+ """
202
+ # ── Preprocess ────────────────────────────────────────────────────────
203
+ tensor = self.preprocessor(image).to(self.device) # (1, 3, 224, 224)
204
+
205
+ # Keep original image for overlay (denormalize for display)
206
+ original_pil = self._tensor_to_pil(tensor[0])
207
+
208
+ # ── Forward pass WITH gradients ───────────────────────────────────────
209
+ # We cannot use torch.inference_mode() here β€” Grad-CAM requires
210
+ # gradients to flow backward through the network.
211
+ # model.eval() is still set so dropout is off.
212
+ tensor.requires_grad_(True)
213
+
214
+ output = self.model(tensor) # {"logits": (1,2), "probs": (1,2)}
215
+ logits = output["logits"] # (1, 2)
216
+ probs = output["probs"] # (1, 2)
217
+
218
+ # ── Determine target class ────────────────────────────────────────────
219
+ if class_idx is None:
220
+ class_idx = int(torch.argmax(probs, dim=1).item())
221
+
222
+ confidence = float(probs[0, class_idx].item())
223
+ prediction = LABEL_MAP[class_idx]
224
+
225
+ # ── Backward pass β€” gradients of class score w.r.t. feature maps ─────
226
+ self.model.zero_grad()
227
+
228
+ # Score for the target class β€” scalar
229
+ class_score = logits[0, class_idx]
230
+ class_score.backward() # fills self._gradients via hook
231
+
232
+ # ── Compute Grad-CAM weights ──────────────────────────────────────────
233
+ # gradients: (1, 1024, 7, 7) β€” how much each spatial location in each
234
+ # channel affected the class score
235
+ # Global average pool the gradients β†’ (1, 1024, 1, 1)
236
+ # This gives the importance weight for each of the 1024 channels
237
+ weights = self._gradients.mean(dim=(2, 3), keepdim=True) # (1, 1024, 1, 1)
238
+
239
+ # activations: (1, 1024, 7, 7) β€” what the layer actually responded to
240
+ # Weighted sum across channels β†’ (1, 1, 7, 7)
241
+ # Each of 1024 channels is multiplied by its importance weight
242
+ cam = (weights * self._activations).sum(dim=1, keepdim=True) # (1, 1, 7, 7)
243
+
244
+ # ── Post-process ──────────────────────────────────────────────────────
245
+ # ReLU: keep only positive contributions β€” negative means the region
246
+ # pushed AGAINST the predicted class, which we don't visualize
247
+ cam = torch.relu(cam)
248
+
249
+ # Convert to numpy, remove batch and channel dims β†’ (7, 7)
250
+ cam_raw = cam[0, 0].cpu().numpy()
251
+
252
+ # Normalize to [0, 1]
253
+ cam_min, cam_max = cam_raw.min(), cam_raw.max()
254
+ if cam_max > cam_min:
255
+ cam_norm = (cam_raw - cam_min) / (cam_max - cam_min)
256
+ else:
257
+ cam_norm = np.zeros_like(cam_raw)
258
+
259
+ # Upsample from 7Γ—7 to 224Γ—224 using PIL (no scipy dependency)
260
+ heatmap = self._upsample_cam(cam_norm, size=(224, 224))
261
+
262
+ # ── Build overlay ─────────────────────────────────────────────────────
263
+ overlay = self._build_overlay(original_pil, heatmap)
264
+
265
+ return {
266
+ "prediction": prediction,
267
+ "confidence": round(confidence, 6),
268
+ "logits": logits.detach().cpu(),
269
+ "heatmap": heatmap, # (224, 224) float [0, 1]
270
+ "overlay": overlay, # PIL.Image
271
+ "cam_raw": cam_raw, # (7, 7) raw before upsample
272
+ }
273
+
274
+ # ── Helpers ───────────────────────────────────────────────────────────────
275
+
276
+ def _upsample_cam(self, cam: np.ndarray, size: tuple[int, int]) -> np.ndarray:
277
+ """
278
+ Upsample a (7, 7) CAM to the target size using PIL bilinear resize.
279
+ Returns a float32 ndarray normalized to [0, 1].
280
+ """
281
+ cam_uint8 = (cam * 255).astype(np.uint8)
282
+ cam_pil = Image.fromarray(cam_uint8, mode="L")
283
+ cam_pil = cam_pil.resize(size, Image.BILINEAR)
284
+ return np.array(cam_pil, dtype=np.float32) / 255.0
285
+
286
+ def _build_overlay(
287
+ self,
288
+ original: "Image.Image",
289
+ heatmap: np.ndarray,
290
+ ) -> "Image.Image":
291
+ """
292
+ Blend the jet-coloured heatmap onto the original image.
293
+
294
+ Parameters
295
+ ----------
296
+ original : PIL.Image β€” original image at 224Γ—224
297
+ heatmap : ndarray β€” (224, 224) float in [0, 1]
298
+
299
+ Returns
300
+ -------
301
+ PIL.Image β€” RGBA blended result
302
+ """
303
+ # Colour the heatmap using jet colourmap
304
+ jet_rgb = _apply_jet_colormap(heatmap) # (224, 224, 3) uint8
305
+ jet_pil = Image.fromarray(jet_rgb, mode="RGB")
306
+
307
+ # Ensure original is RGB
308
+ original_rgb = original.convert("RGB")
309
+
310
+ # Blend: overlay = (1 - alpha) * original + alpha * heatmap
311
+ overlay = Image.blend(original_rgb, jet_pil, alpha=self.alpha)
312
+ return overlay
313
+
314
+ @staticmethod
315
+ def _tensor_to_pil(tensor: torch.Tensor) -> "Image.Image":
316
+ """
317
+ Convert a (3, H, W) normalized tensor back to a PIL Image for display.
318
+ Reverses ImageNet normalization so the image looks natural.
319
+ """
320
+ # ImageNet mean / std β€” same constants as preprocessing.py
321
+ mean = torch.tensor([0.485, 0.456, 0.406]).view(3, 1, 1)
322
+ std = torch.tensor([0.229, 0.224, 0.225]).view(3, 1, 1)
323
+
324
+ # Denormalize: x_original = x_normalized * std + mean
325
+ img = tensor.detach().cpu() * std + mean
326
+ img = img.clamp(0, 1)
327
+
328
+ # Convert to uint8 PIL Image
329
+ arr = (img.permute(1, 2, 0).numpy() * 255).astype(np.uint8)
330
+ return Image.fromarray(arr, mode="RGB")
331
+
332
+ @staticmethod
333
+ def _resolve_device(device: Optional[str]) -> torch.device:
334
+ if device is not None:
335
+ return torch.device(device)
336
+ if torch.cuda.is_available():
337
+ return torch.device("cuda")
338
+ if torch.backends.mps.is_available():
339
+ return torch.device("mps")
340
+ return torch.device("cpu")
explainability/llm_chat.py ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ explainability/llm_chat.py
3
+ ───────────────────────────
4
+ A real conversational LLM for the platform's AI assistant β€” defaults to
5
+ GROQ (free, no credit card), with Anthropic / OpenAI as optional providers.
6
+ Streams tokens so the assistant behaves like ChatGPT, while staying grounded
7
+ in this platform's models, metrics, and the current scan context.
8
+
9
+ Why Groq
10
+ ────────
11
+ Groq's free tier serves open-source models (Llama 3.3 70B etc.) on very
12
+ fast LPU hardware, with no credit card. It's OpenAI-API-compatible, so it
13
+ works as a drop-in via the OpenAI SDK pointed at Groq's base URL. Unlike
14
+ Ollama it needs no GPU server of your own β€” ideal for a deployed website.
15
+
16
+ Configuration (environment variables)
17
+ ──────────────────────────────────────
18
+ CHAT_PROVIDER "groq" (default) | "anthropic" | "openai"
19
+ CHAT_MODEL model id. Defaults:
20
+ groq β†’ llama-3.3-70b-versatile (ChatGPT-quality)
21
+ (use llama-3.1-8b-instant for higher limits)
22
+ anthropic β†’ claude-3-5-haiku-latest
23
+ openai β†’ gpt-4o-mini
24
+ GROQ_API_KEY required for groq (console.groq.com β€” free)
25
+ ANTHROPIC_API_KEY required for anthropic (paid)
26
+ OPENAI_API_KEY required for openai (paid)
27
+
28
+ Install the SDK (Groq uses the OpenAI SDK):
29
+ pip install openai # for groq or openai
30
+ pip install anthropic # only if you choose anthropic
31
+
32
+ Deployment
33
+ ──────────
34
+ Set GROQ_API_KEY as an environment variable / secret on your host
35
+ (Render, Railway, Fly, a VPS, etc.). NEVER put the key in the frontend.
36
+ """
37
+
38
+ from __future__ import annotations
39
+
40
+ import os
41
+ from typing import Iterator
42
+
43
+ GROQ_BASE_URL = "https://api.groq.com/openai/v1"
44
+
45
+ SYSTEM_PROMPT = """You are MedAI's assistant β€” a sharp, articulate expert in medical-imaging AI, deep learning, and radiology, with broad general knowledge. You think clearly and engage like a thoughtful senior colleague (in the spirit of ChatGPT or Claude): you reason through problems, give intuition and concrete examples, use analogies, structure substantial answers with headers and lists, and anticipate the next question.
46
+
47
+ You live inside MedAI, a research platform for breast-cancer detection from medical images. When asked about "this platform", ground answers in these facts:
48
+ - Histopathology: a DenseNet-121 CNN trained on PatchCamelyon (PCam) β€” H&E-stained lymph-node patches. ~88.0% accuracy, 87.5% sensitivity. Explained with Grad-CAM.
49
+ - Mammography: a 3-model EfficientNet-B4 ensemble trained on RSNA 2022. 0.84 AUC, 70.1% sensitivity, 82.4% specificity on the RSNA validation set. Explained with Grad-CAM.
50
+ - An experiment adding breast-ROI cropping + external CBIS-DDSM data reduced performance (cross-domain film-vs-digital shift plus overfitting), so the platform kept the un-cropped RSNA-only ensemble.
51
+
52
+ You are NOT limited to reciting those facts. Discuss the underlying ML and medicine in real depth β€” architectures, training methods, loss functions, class imbalance, metrics (AUC, sensitivity/specificity, calibration, operating points), Grad-CAM, BI-RADS, dataset pitfalls, external validation, regulatory paths, deployment β€” and answer broader questions intelligently. Bring genuine insight: explain *why*, not just *what*.
53
+
54
+ Style:
55
+ - You are in a CHAT panel, not a document. Keep replies conversational and reasonably brief β€” usually a few short paragraphs, with bullet or numbered lists where they genuinely help. Expand into depth only when the question truly calls for it.
56
+ - Do NOT use markdown tables. The chat panel is narrow (often mobile) and cannot render them β€” they come out as garbled pipes. Convey the same information as short paragraphs or bullet lists instead.
57
+ - Use light markdown only: **bold**, bullet/numbered lists, and `inline code`. Use ## headers sparingly and only in genuinely long answers. Don't turn a chat reply into a multi-section report with FAQs unless the user explicitly asks for that.
58
+ - Be direct and concrete; explain the "why" and name tradeoffs rather than padding.
59
+
60
+ Safety (non-negotiable):
61
+ - This is a research/educational tool β€” NOT a medical device, NOT a diagnosis. Never tell anyone they do or don't have cancer, and never give definitive clinical advice; a qualified clinician makes the call. Be honest about limits (e.g. 70% mammogram sensitivity means roughly 1 in 3 cancers can be missed).
62
+ - Do not treat the model's confidence score as a calibrated probability of being correct β€” the model is not calibrated. Say it "leans toward" a label or is "not certain", rather than quoting the percentage as a literal chance of error (e.g. don't say "76% confidence means a 24% chance it's wrong").
63
+ - Adapt tone to the audience when indicated β€” clinician (clinical framing, BI-RADS, workup), researcher (ML/technical depth), patient (plain, calm, reassuring; encourage seeing their doctor) β€” while staying clear and professional.
64
+ """
65
+
66
+
67
+ def _default_model(provider: str) -> str:
68
+ return {
69
+ "groq": "openai/gpt-oss-120b",
70
+ "anthropic": "claude-3-5-haiku-latest",
71
+ "openai": "gpt-4o-mini",
72
+ }.get(provider, "openai/gpt-oss-120b")
73
+
74
+
75
+ def llm_available() -> tuple[bool, str]:
76
+ """Return (available, provider) β€” whether an API key is configured."""
77
+ provider = os.getenv("CHAT_PROVIDER", "groq").lower()
78
+ key = {
79
+ "groq": "GROQ_API_KEY",
80
+ "anthropic": "ANTHROPIC_API_KEY",
81
+ "openai": "OPENAI_API_KEY",
82
+ }.get(provider, "GROQ_API_KEY")
83
+ return (bool(os.getenv(key)), provider)
84
+
85
+
86
+ def _context_block(context: dict | None, audience: str) -> str:
87
+ lines = [f"Audience mode: {audience}."]
88
+ if context:
89
+ pred = context.get("prediction")
90
+ if pred:
91
+ conf = context.get("confidence", 0.0)
92
+ lines.append(
93
+ f"Current scan in view β€” model prediction: {pred} "
94
+ f"({conf*100:.1f}% confidence)."
95
+ )
96
+ logits = context.get("logits") or []
97
+ if len(logits) >= 2:
98
+ lines.append(f"Raw logits: benign={logits[0]:.3f}, malignant={logits[1]:.3f}.")
99
+ spatial = context.get("spatial_summary")
100
+ if spatial:
101
+ lines.append(f"Grad-CAM spatial finding: {spatial}")
102
+ birads = context.get("birads")
103
+ if birads:
104
+ lines.append(f"Suggested BI-RADS: {birads}.")
105
+ else:
106
+ lines.append("No scan is currently loaded; answer about the platform or general concepts.")
107
+ return "\n".join(lines)
108
+
109
+
110
+ def _openai_compatible_stream(system: str, messages: list[dict], model: str,
111
+ base_url: str | None, api_key_env: str) -> Iterator[str]:
112
+ """Shared streamer for OpenAI-compatible endpoints (OpenAI and Groq)."""
113
+ try:
114
+ from openai import OpenAI
115
+ except ImportError:
116
+ yield "The OpenAI SDK isn't installed. Run: pip install openai"
117
+ return
118
+ try:
119
+ kwargs = {}
120
+ if base_url:
121
+ kwargs["base_url"] = base_url
122
+ kwargs["api_key"] = os.getenv(api_key_env)
123
+ client = OpenAI(**kwargs)
124
+ stream = client.chat.completions.create(
125
+ model=model,
126
+ messages=[{"role": "system", "content": system}] + messages,
127
+ max_tokens=2048, temperature=0.6, stream=True,
128
+ )
129
+ for chunk in stream:
130
+ delta = chunk.choices[0].delta.content
131
+ if delta:
132
+ yield delta
133
+ except Exception as e: # noqa: BLE001
134
+ yield f"\n[LLM error: {e}]"
135
+
136
+
137
+ def _anthropic_stream(system: str, messages: list[dict], model: str) -> Iterator[str]:
138
+ try:
139
+ import anthropic
140
+ except ImportError:
141
+ yield "The Anthropic SDK isn't installed. Run: pip install anthropic"
142
+ return
143
+ try:
144
+ client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
145
+ with client.messages.stream(
146
+ model=model, max_tokens=1024, system=system, messages=messages,
147
+ ) as stream:
148
+ for text in stream.text_stream:
149
+ yield text
150
+ except Exception as e: # noqa: BLE001
151
+ yield f"\n[LLM error: {e}]"
152
+
153
+
154
+ def stream_chat(
155
+ messages: list[dict],
156
+ context: dict | None = None,
157
+ audience: str = "clinician",
158
+ ) -> Iterator[str]:
159
+ """
160
+ Stream an assistant reply token-by-token.
161
+
162
+ messages: list of {"role": "user"|"assistant", "content": str} (history,
163
+ ending with the latest user turn).
164
+ context: optional dict with prediction/confidence/logits/spatial_summary/birads.
165
+ """
166
+ available, provider = llm_available()
167
+ if not available:
168
+ key = {"groq": "GROQ_API_KEY", "anthropic": "ANTHROPIC_API_KEY",
169
+ "openai": "OPENAI_API_KEY"}.get(provider, "GROQ_API_KEY")
170
+ yield (f"The AI assistant needs an API key. Get a free one at "
171
+ f"console.groq.com, then set {key} in the environment and restart "
172
+ f"the API (CHAT_PROVIDER defaults to 'groq').")
173
+ return
174
+
175
+ system = SYSTEM_PROMPT + "\n\n" + _context_block(context, audience)
176
+ clean = [{"role": m["role"], "content": str(m["content"])}
177
+ for m in messages if m.get("role") in ("user", "assistant") and m.get("content")]
178
+ if not clean:
179
+ clean = [{"role": "user", "content": "Hello"}]
180
+
181
+ model = os.getenv("CHAT_MODEL", _default_model(provider))
182
+
183
+ if provider == "anthropic":
184
+ yield from _anthropic_stream(system, clean, model)
185
+ elif provider == "openai":
186
+ yield from _openai_compatible_stream(system, clean, model, None, "OPENAI_API_KEY")
187
+ else: # groq (default)
188
+ yield from _openai_compatible_stream(system, clean, model, GROQ_BASE_URL, "GROQ_API_KEY")
explainability/llm_explain.py ADDED
@@ -0,0 +1,1125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ explainability/llm_explain.py
3
+ ──────────────────────────────
4
+ Local LLM-powered natural language explanation of model predictions.
5
+ Built from scratch using HuggingFace Transformers β€” no external API,
6
+ no API key, runs entirely on your machine.
7
+
8
+ Architecture
9
+ ────────────
10
+ Tokenizer : AutoTokenizer (google/flan-t5-base)
11
+ Model : T5ForConditionalGeneration
12
+ Inference : beam search, greedy decode, or sampling
13
+ Fallback : deterministic template engine (works with no model at all)
14
+
15
+ FLAN-T5 was chosen because:
16
+ - Instruction-tuned β†’ responds well to structured prompts
17
+ - No API key needed β†’ fully self-contained
18
+ - Reasonable size β†’ flan-t5-base is ~250 MB, runs on CPU
19
+ - Medical text β†’ handles clinical terminology cleanly
20
+ - You've used it before in RialoLens AI Explainer Engine
21
+
22
+ Model variants (pass as model_name)
23
+ ────────────────────────────────────
24
+ "google/flan-t5-small" ~80 MB fastest, lowest quality
25
+ "google/flan-t5-base" ~250 MB ← default, good balance
26
+ "google/flan-t5-large" ~780 MB better quality, needs more RAM
27
+ "google/flan-t5-xl" ~3 GB best quality, needs GPU
28
+
29
+ How it connects to the existing architecture
30
+ ────────────────────────────────────────────
31
+ model/inference.py β†’ consumes its prediction dict output
32
+ explainability/gradcam.py β†’ optionally consumes Grad-CAM result dict
33
+ No existing files are modified.
34
+
35
+ Usage
36
+ ─────
37
+ from model.inference import BreastCancerInferencePipeline
38
+ from explainability import GradCAM, LLMExplainer
39
+
40
+ pipeline = BreastCancerInferencePipeline("model/weights.pth")
41
+ result = pipeline.predict("slide.png")
42
+
43
+ # Basic explanation
44
+ llm = LLMExplainer() # downloads model once
45
+ report = llm.explain(result, audience="clinician")
46
+ print(report["summary"])
47
+
48
+ # With Grad-CAM spatial context
49
+ cam = GradCAM(pipeline.model)
50
+ cam_result = cam.explain("slide.png")
51
+ report = llm.explain_with_gradcam(cam_result, audience="patient")
52
+ print(report["detail"])
53
+
54
+ Install
55
+ ───────
56
+ pip install transformers sentencepiece accelerate
57
+ """
58
+
59
+ from __future__ import annotations
60
+
61
+ import sys
62
+ import textwrap
63
+ from pathlib import Path
64
+ from typing import Literal, Optional
65
+
66
+ ROOT = Path(__file__).resolve().parents[1]
67
+ sys.path.insert(0, str(ROOT))
68
+
69
+ # Audience type
70
+ Audience = Literal["clinician", "researcher", "patient"]
71
+
72
+
73
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
74
+ # β•‘ FLAN-T5 BACKBONE β•‘
75
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
76
+
77
+ class FlanT5Engine:
78
+ """
79
+ Thin wrapper around FLAN-T5 for text generation.
80
+
81
+ Downloads the model once on first use and caches it to
82
+ ~/.cache/huggingface/hub (HuggingFace default).
83
+
84
+ Parameters
85
+ ----------
86
+ model_name : str
87
+ HuggingFace model identifier. Default: google/flan-t5-base
88
+ device : str
89
+ "cuda", "mps", or "cpu". Auto-detected if None.
90
+ max_new_tokens : int
91
+ Maximum tokens to generate per explanation. Default 256.
92
+ """
93
+
94
+ def __init__(
95
+ self,
96
+ model_name: str = "google/flan-t5-large",
97
+ device: Optional[str] = None,
98
+ max_new_tokens: int = 256,
99
+ ) -> None:
100
+ self.model_name = model_name
101
+ self.max_new_tokens = max_new_tokens
102
+ self.device = self._resolve_device(device)
103
+
104
+ self._tokenizer = None
105
+ self._model = None
106
+
107
+ # ── Lazy loading β€” model loads on first generate() call ──────────────────
108
+
109
+ def _load(self) -> None:
110
+ """Download / load tokenizer and model into memory."""
111
+ if self._model is not None:
112
+ return # already loaded
113
+
114
+ try:
115
+ from transformers import AutoTokenizer, T5ForConditionalGeneration
116
+ except ImportError:
117
+ raise ImportError(
118
+ "transformers not installed.\n"
119
+ "Run: pip install transformers sentencepiece"
120
+ )
121
+
122
+ import torch
123
+
124
+ print(f"[LLMExplainer] Loading {self.model_name} …")
125
+ print(f"[LLMExplainer] Device: {self.device}")
126
+ print("[LLMExplainer] First run downloads ~250 MB, cached afterwards.")
127
+
128
+ self._tokenizer = AutoTokenizer.from_pretrained(self.model_name)
129
+
130
+ self._model = T5ForConditionalGeneration.from_pretrained(
131
+ self.model_name,
132
+ torch_dtype=torch.float16 if self.device != "cpu" else torch.float32,
133
+ ).to(self.device)
134
+
135
+ self._model.eval()
136
+ print(f"[LLMExplainer] Model ready.")
137
+
138
+ def generate(self, prompt: str) -> str:
139
+ """
140
+ Generate text from a prompt using FLAN-T5.
141
+
142
+ Parameters
143
+ ----------
144
+ prompt : str
145
+ Instruction-style prompt in the T5 format.
146
+
147
+ Returns
148
+ -------
149
+ str β€” generated text, decoded and stripped.
150
+ """
151
+ import torch
152
+
153
+ self._load()
154
+
155
+ inputs = self._tokenizer(
156
+ prompt,
157
+ return_tensors="pt",
158
+ truncation=True,
159
+ max_length=512, # T5 input limit
160
+ ).to(self.device)
161
+
162
+ with torch.inference_mode():
163
+ output_ids = self._model.generate(
164
+ **inputs,
165
+ max_new_tokens = self.max_new_tokens,
166
+ num_beams = 4, # beam search β€” better quality
167
+ early_stopping = True,
168
+ no_repeat_ngram_size = 3, # reduce repetition
169
+ length_penalty = 1.2, # encourage longer outputs
170
+ )
171
+
172
+ decoded = self._tokenizer.decode(
173
+ output_ids[0],
174
+ skip_special_tokens=True,
175
+ )
176
+ return decoded.strip()
177
+
178
+ def generate_chat(self, prompt: str) -> str:
179
+ """
180
+ Generate a conversational chat response using FLAN-T5.
181
+
182
+ Uses temperature sampling instead of beam search to produce
183
+ natural, varied, human-sounding responses rather than mechanical outputs.
184
+
185
+ Parameters
186
+ ----------
187
+ prompt : str
188
+ Conversational prompt with full context and instruction.
189
+
190
+ Returns
191
+ -------
192
+ str β€” generated text, decoded and stripped.
193
+ """
194
+ import torch
195
+
196
+ self._load()
197
+
198
+ inputs = self._tokenizer(
199
+ prompt,
200
+ return_tensors = "pt",
201
+ truncation = True,
202
+ max_length = 512,
203
+ ).to(self.device)
204
+
205
+ with torch.inference_mode():
206
+ output_ids = self._model.generate(
207
+ **inputs,
208
+ max_new_tokens = 400,
209
+ do_sample = True, # sampling for natural variety
210
+ temperature = 0.8, # slightly creative but still coherent
211
+ top_p = 0.92, # nucleus sampling
212
+ no_repeat_ngram_size = 3,
213
+ repetition_penalty = 1.3,
214
+ )
215
+
216
+ decoded = self._tokenizer.decode(
217
+ output_ids[0],
218
+ skip_special_tokens = True,
219
+ )
220
+ return decoded.strip()
221
+
222
+ @staticmethod
223
+ def _resolve_device(device: Optional[str]) -> str:
224
+ import torch
225
+ if device is not None:
226
+ return device
227
+ if torch.cuda.is_available():
228
+ return "cuda"
229
+ if torch.backends.mps.is_available():
230
+ return "mps"
231
+ return "cpu"
232
+
233
+
234
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
235
+ # β•‘ TEMPLATE ENGINE β€” deterministic fallback β•‘
236
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
237
+
238
+ class TemplateEngine:
239
+ """
240
+ Audience-aware deterministic explanation engine.
241
+ Produces genuinely different content for clinician, researcher, and patient.
242
+ """
243
+
244
+ TIERS = [
245
+ (0.95, "very high", "very high confidence"),
246
+ (0.85, "high", "high confidence"),
247
+ (0.70, "moderate", "moderate confidence"),
248
+ (0.55, "borderline", "borderline confidence β€” treat with caution"),
249
+ (0.00, "low", "low confidence β€” result unreliable without further review"),
250
+ ]
251
+
252
+ def confidence_tier(self, conf: float) -> tuple[str, str]:
253
+ for threshold, label, desc in self.TIERS:
254
+ if conf >= threshold:
255
+ return label, desc
256
+ return "low", "low confidence"
257
+
258
+ # Modality-specific facts so explanations name the right model, dataset,
259
+ # reviewer, and sample type. Histopathology = DenseNet/PCam/pathologist;
260
+ # mammogram = EfficientNet-B4 ensemble/RSNA/radiologist.
261
+ MODALITY = {
262
+ "histopathology": {
263
+ "model": "DenseNet-121",
264
+ "sample": "patch",
265
+ "sample_plain": "tissue sample",
266
+ "metrics": "87.5% sensitivity and 88.0% accuracy on the held-out PCam test set",
267
+ "reviewer": "pathologist",
268
+ "correlation": "histomorphological correlation",
269
+ "pos_action": "tissue biopsy and full clinical assessment",
270
+ "neg_action": "Routine 12-month follow-up is appropriate if clinically indicated.",
271
+ "imaging_desc": "checks tissue under a microscope",
272
+ "abnormal": "abnormal cells",
273
+ "model_full": "DenseNet-121 (7.22M params) fine-tuned on 220,025 "
274
+ "deduplicated PCam patches",
275
+ "train_detail": "OneCycleLR (max_lr=3e-3), Mixup (Ξ±=0.4), StainJitter "
276
+ "(HED, strength=0.05), label smoothing (0.1), "
277
+ "CrossEntropyLoss (pos_weight=1.469)",
278
+ "metrics_short":"Best test sensitivity: 87.5%, accuracy: 88.0%",
279
+ },
280
+ "mammogram": {
281
+ "model": "the EfficientNet-B4 ensemble",
282
+ "sample": "mammogram",
283
+ "sample_plain": "mammogram",
284
+ "metrics": "0.84 AUC (70.1% sensitivity, 82.4% specificity) on the "
285
+ "RSNA validation set",
286
+ "reviewer": "radiologist",
287
+ "correlation": "radiological correlation",
288
+ "pos_action": "a diagnostic mammographic work-up and possible biopsy",
289
+ "neg_action": "Routine screening follow-up is appropriate per BI-RADS guidance.",
290
+ "imaging_desc": "reviews the breast X-ray image",
291
+ "abnormal": "an abnormal area",
292
+ "model_full": "a 3-model EfficientNet-B4 ensemble trained on the RSNA 2022 "
293
+ "mammography dataset (54,706 images)",
294
+ "train_detail": "3Γ— EfficientNet-B4 (seeds 42/123/999), WeightedRandomSampler "
295
+ "for class balance, AMP mixed precision, CrossEntropyLoss "
296
+ "(weight=[1,5]), predictions averaged across members",
297
+ "metrics_short":"Ensemble AUC: 0.84 (sensitivity 70.1%, specificity 82.4%)",
298
+ },
299
+ }
300
+
301
+ def build(
302
+ self,
303
+ prediction: str,
304
+ confidence: float,
305
+ benign_logit: float,
306
+ malignant_logit: float,
307
+ audience: Audience,
308
+ gradcam_context: Optional[str] = None,
309
+ modality: str = "histopathology",
310
+ birads: Optional[str] = None,
311
+ ) -> tuple[str, str]:
312
+ tier_label, tier_desc = self.confidence_tier(confidence)
313
+ pct = f"{confidence:.1%}"
314
+ margin = abs(malignant_logit - benign_logit)
315
+ is_mal = prediction == "malignant"
316
+ cam = gradcam_context or ""
317
+ f = self.MODALITY.get(modality, self.MODALITY["histopathology"])
318
+
319
+ # ── CLINICIAN ─────────────────────────────────────────────────────────
320
+ if audience == "clinician":
321
+ # Use the caller-supplied BI-RADS (from the ensemble) when available,
322
+ # otherwise fall back to a confidence-derived suggestion.
323
+ birad = birads or (
324
+ "BI-RADS 4B β€” Suspicious" if is_mal and confidence >= 0.75 else
325
+ "BI-RADS 4A β€” Low suspicion" if is_mal else
326
+ "BI-RADS 2 β€” Benign finding"
327
+ )
328
+ boundary = (
329
+ "The decision margin of {:.3f} indicates a clear separation "
330
+ "from the decision boundary, supporting diagnostic reliability.".format(margin)
331
+ if margin > 1.5 else
332
+ "The decision margin of {:.3f} places this case near the "
333
+ "classification boundary β€” a borderline result requiring careful "
334
+ "{}.".format(margin, f["correlation"])
335
+ )
336
+ cam_line = (
337
+ f" Grad-CAM spatial analysis: {cam}"
338
+ if cam else
339
+ " Grad-CAM spatial attention maps are available for region-level review."
340
+ )
341
+ summary = (
342
+ f"{f['model']} classified this {f['sample']} as "
343
+ f"{'MALIGNANT' if is_mal else 'BENIGN'} at {pct} ({tier_desc}). "
344
+ f"Raw logits: benign = {benign_logit:.4f}, malignant = {malignant_logit:.4f} "
345
+ f"(margin: {margin:.4f}). "
346
+ f"Suggested {birad}."
347
+ )
348
+ detail = (
349
+ f"{boundary}"
350
+ f"{cam_line} "
351
+ f"Underlying model performance: {f['metrics']}. "
352
+ f"{('A positive result warrants ' + f['pos_action'] + '.') if is_mal else f['neg_action']} "
353
+ f"This output is AI-assisted and must not replace {f['reviewer']} review."
354
+ )
355
+
356
+ # ── RESEARCHER ────────────────────────────────────────────────────────
357
+ elif audience == "researcher":
358
+ softmax_b = round(1 / (1 + 2.718 ** (malignant_logit - benign_logit)), 4)
359
+ softmax_m = round(1 - softmax_b, 4)
360
+ cam_line = f" Grad-CAM activation summary: {cam}" if cam else ""
361
+ summary = (
362
+ f"Classification output: {prediction.upper()} "
363
+ f"[softmax({benign_logit:.4f}, {malignant_logit:.4f}) = "
364
+ f"({softmax_b:.4f}, {softmax_m:.4f})]. "
365
+ f"Argmax class = {'1 (malignant)' if is_mal else '0 (benign)'}. "
366
+ f"Decision margin |Ξ”logit| = {margin:.4f} "
367
+ f"({'above' if margin > 1.5 else 'below'} the 1.5 heuristic threshold "
368
+ f"for high-confidence separation)."
369
+ )
370
+ detail = (
371
+ f"Model: {f['model_full']}. Training: {f['train_detail']}. "
372
+ f"{f['metrics_short']}. "
373
+ f"Softmax probabilities are uncalibrated β€” no temperature scaling applied."
374
+ f"{cam_line}"
375
+ )
376
+
377
+ # ── PATIENT ───────────────────────────────────────────────────────────
378
+ else:
379
+ sample = f["sample_plain"]
380
+ if is_mal:
381
+ if confidence >= 0.85:
382
+ summary = (
383
+ f"The AI system flagged an area in this {sample} that looks "
384
+ f"unusual, and it is fairly confident about this ({pct}). "
385
+ f"This is a signal that a doctor should take a closer look β€” "
386
+ f"it does not mean you definitely have cancer."
387
+ )
388
+ detail = (
389
+ f"Think of this AI like a second set of eyes that {f['imaging_desc']}. "
390
+ f"It spotted a pattern it associates with {f['abnormal']} in this {sample}. "
391
+ f"{'The AI was also looking at the ' + cam.split('.')[0].lower() + ' area most closely.' if cam else ''} "
392
+ f"Your doctor will review this result and decide the right next step β€” "
393
+ f"this might be a follow-up scan or a biopsy. Please do not worry "
394
+ f"until you have spoken with your healthcare provider. "
395
+ f"This AI tool is for screening only, not a final diagnosis."
396
+ )
397
+ else:
398
+ summary = (
399
+ f"The AI system found something in this {sample} that it "
400
+ f"wasn't entirely sure about ({pct} confidence). "
401
+ f"The result is uncertain and will need your doctor's review "
402
+ f"before any conclusions are drawn."
403
+ )
404
+ detail = (
405
+ f"This means the AI could not clearly decide whether the {sample} "
406
+ f"looks normal or abnormal β€” it is on the borderline. "
407
+ f"This happens sometimes with difficult cases. "
408
+ f"Your doctor is the right person to interpret this alongside "
409
+ f"your full medical history and any other tests. "
410
+ f"This AI tool is a screening aid only, not a diagnosis."
411
+ )
412
+ else:
413
+ summary = (
414
+ f"The AI system found no signs of an abnormality in this {sample} "
415
+ f"({pct} confidence). This is a reassuring result."
416
+ )
417
+ detail = (
418
+ f"The AI {f['imaging_desc']} and did not find features it associates "
419
+ f"with cancer. "
420
+ f"This is a good sign, but all AI results should be confirmed by "
421
+ f"your doctor as part of your complete care. "
422
+ f"{'The AI was paying attention to ' + cam.split('.')[0].lower() + '.' if cam else ''} "
423
+ f"Please keep any follow-up appointments your doctor recommends. "
424
+ f"This AI tool is for screening only, not a final diagnosis."
425
+ )
426
+
427
+ detail = detail.strip()
428
+ return summary, detail
429
+
430
+
431
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
432
+ # β•‘ PROMPT BUILDER β•‘
433
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
434
+
435
+ class PromptBuilder:
436
+ """
437
+ Builds FLAN-T5 instruction prompts from structured prediction data.
438
+
439
+ FLAN-T5 responds best to explicit task instructions in the format:
440
+ "Task description: [context]. Answer:"
441
+ """
442
+
443
+ AUDIENCE_CONTEXT = {
444
+ "clinician": (
445
+ "a clinical pathologist who needs precise technical details, "
446
+ "logit scores, confidence calibration, and clinical caveats"
447
+ ),
448
+ "researcher": (
449
+ "an ML researcher who wants to understand the model's decision "
450
+ "in terms of logit scores, softmax probabilities, and feature analysis"
451
+ ),
452
+ "patient": (
453
+ "a patient with no medical background who needs a clear, "
454
+ "compassionate explanation without jargon"
455
+ ),
456
+ }
457
+
458
+ def build(
459
+ self,
460
+ prediction: str,
461
+ confidence: float,
462
+ benign_logit: float,
463
+ malignant_logit: float,
464
+ audience: Audience,
465
+ gradcam_context: Optional[str] = None,
466
+ ) -> str:
467
+ """Construct the instruction prompt for FLAN-T5."""
468
+
469
+ tier = (
470
+ "high" if confidence >= 0.85 else
471
+ "moderate" if confidence >= 0.70 else
472
+ "low"
473
+ )
474
+
475
+ cam_section = (
476
+ f" The Grad-CAM spatial analysis shows: {gradcam_context}"
477
+ if gradcam_context else ""
478
+ )
479
+
480
+ prompt = textwrap.dedent(f"""
481
+ Explain a breast cancer AI classifier result to {self.AUDIENCE_CONTEXT[audience]}.
482
+
483
+ Model result:
484
+ - Prediction: {prediction.upper()}
485
+ - Confidence: {confidence:.1%} ({tier} confidence)
486
+ - Benign logit score: {benign_logit:.3f}
487
+ - Malignant logit score: {malignant_logit:.3f}{cam_section}
488
+
489
+ Write a clear 3-sentence explanation of what this result means,
490
+ what the confidence level implies, and remind the reader this is
491
+ a research tool that requires clinical confirmation.
492
+
493
+ Explanation:
494
+ """).strip()
495
+
496
+ return prompt
497
+
498
+
499
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
500
+ # β•‘ MAIN EXPLAINER CLASS β•‘
501
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
502
+
503
+ class LLMExplainer:
504
+ """
505
+ Local LLM-powered natural language explainer for breast cancer predictions.
506
+
507
+ Uses FLAN-T5 running entirely on your machine β€” no API key, no internet
508
+ connection required after the initial model download.
509
+
510
+ Falls back to a deterministic template engine if:
511
+ - use_llm=False is passed
512
+ - transformers is not installed
513
+ - The model fails to generate meaningful output
514
+
515
+ Parameters
516
+ ----------
517
+ model_name : str
518
+ HuggingFace FLAN-T5 variant. Default: google/flan-t5-base (~250 MB).
519
+ device : str | None
520
+ "cuda", "mps", or "cpu". Auto-detected if None.
521
+ max_new_tokens : int
522
+ Max tokens per generated explanation. Default 256.
523
+ use_llm : bool
524
+ Set False to skip FLAN-T5 and use the template engine directly.
525
+ Useful for fast testing or environments without GPU/internet.
526
+ """
527
+
528
+ DISCLAIMER = (
529
+ "Research and educational use only. "
530
+ "Not a standalone diagnostic tool. "
531
+ "Clinical confirmation by a qualified pathologist is required."
532
+ )
533
+
534
+ def __init__(
535
+ self,
536
+ model_name: str = "google/flan-t5-large",
537
+ device: Optional[str] = None,
538
+ max_new_tokens: int = 256,
539
+ use_llm: bool = True,
540
+ ) -> None:
541
+ self.use_llm = use_llm
542
+ self._prompt_builder = PromptBuilder()
543
+ self._template = TemplateEngine()
544
+
545
+ if use_llm:
546
+ self._llm = FlanT5Engine(
547
+ model_name = model_name,
548
+ device = device,
549
+ max_new_tokens = max_new_tokens,
550
+ )
551
+ else:
552
+ self._llm = None
553
+ print("[LLMExplainer] Running in template-only mode (use_llm=False).")
554
+
555
+ # ── Public API ────────────────────────────────────────────────────────────
556
+
557
+ def explain(
558
+ self,
559
+ prediction: dict,
560
+ audience: Audience = "clinician",
561
+ modality: str = "histopathology",
562
+ ) -> dict:
563
+ """
564
+ Generate a natural language explanation from an inference.py output dict.
565
+
566
+ Parameters
567
+ ----------
568
+ prediction : dict
569
+ Output from a predict() call:
570
+ {"prediction": str, "confidence": float, "logits": Tensor[1,2],
571
+ "birads": str (optional)}
572
+ audience : "clinician" | "researcher" | "patient"
573
+ modality : "histopathology" | "mammogram"
574
+
575
+ Returns
576
+ -------
577
+ dict
578
+ {
579
+ "summary" : str β€” plain-language summary
580
+ "detail" : str β€” deeper explanation with confidence context
581
+ "disclaimer" : str β€” standard research disclaimer
582
+ "audience" : str β€” target audience
583
+ "engine" : str β€” "flan-t5" | "template"
584
+ }
585
+ """
586
+ pred, conf, b_logit, m_logit = self._unpack(prediction)
587
+ birads = prediction.get("birads") if isinstance(prediction, dict) else None
588
+ return self._generate(pred, conf, b_logit, m_logit, audience,
589
+ gradcam_context=None, modality=modality, birads=birads)
590
+
591
+ def explain_with_gradcam(
592
+ self,
593
+ gradcam_result: dict,
594
+ audience: Audience = "clinician",
595
+ modality: str = "histopathology",
596
+ ) -> dict:
597
+ """
598
+ Generate an explanation that incorporates Grad-CAM spatial findings.
599
+
600
+ Parameters
601
+ ----------
602
+ gradcam_result : dict
603
+ Output from a GradCAM.explain() call:
604
+ {"prediction", "confidence", "logits", "heatmap", "birads"(optional), ...}
605
+ audience : "clinician" | "researcher" | "patient"
606
+ modality : "histopathology" | "mammogram"
607
+
608
+ Returns
609
+ -------
610
+ Same schema as explain() β€” adds spatial activation context.
611
+ """
612
+ pred, conf, b_logit, m_logit = self._unpack(gradcam_result)
613
+ gradcam_context = self._summarise_heatmap(gradcam_result["heatmap"])
614
+ birads = gradcam_result.get("birads") if isinstance(gradcam_result, dict) else None
615
+ return self._generate(pred, conf, b_logit, m_logit, audience,
616
+ gradcam_context=gradcam_context, modality=modality,
617
+ birads=birads)
618
+
619
+ # ── Internal generation flow ──────────────────────────────────────────────
620
+
621
+ def _generate(
622
+ self,
623
+ prediction: str,
624
+ confidence: float,
625
+ benign_logit: float,
626
+ malignant_logit: float,
627
+ audience: Audience,
628
+ gradcam_context: Optional[str],
629
+ modality: str = "histopathology",
630
+ birads: Optional[str] = None,
631
+ ) -> dict:
632
+ """
633
+ Core generation method. Tries FLAN-T5 first, falls back to template.
634
+ """
635
+ summary, detail = self._template.build(
636
+ prediction = prediction,
637
+ confidence = confidence,
638
+ benign_logit = benign_logit,
639
+ malignant_logit = malignant_logit,
640
+ audience = audience,
641
+ gradcam_context = gradcam_context,
642
+ modality = modality,
643
+ birads = birads,
644
+ )
645
+ engine_used = "template"
646
+
647
+ # ── Optional FLAN-T5 enhancement ─────────────────────────────────────
648
+ if self.use_llm and self._llm is not None:
649
+ try:
650
+ prompt = self._prompt_builder.build(
651
+ prediction = prediction,
652
+ confidence = confidence,
653
+ benign_logit = benign_logit,
654
+ malignant_logit = malignant_logit,
655
+ audience = audience,
656
+ gradcam_context = gradcam_context,
657
+ )
658
+ generated = self._llm.generate(prompt)
659
+ if len(generated.split()) >= 20:
660
+ # Append FLAN-T5 output to template detail β€” don't replace it
661
+ detail = detail + " " + generated.strip()
662
+ engine_used = "flan-t5"
663
+ except Exception as e:
664
+ print(f"[LLMExplainer] FLAN-T5 enhancement failed ({e}) β€” template only.")
665
+
666
+ return {
667
+ "summary": summary,
668
+ "detail": detail,
669
+ "disclaimer": self.DISCLAIMER,
670
+ "audience": audience,
671
+ "engine": engine_used,
672
+ }
673
+
674
+ # ── Helpers ───────────────────────────────────────────────────────────────
675
+
676
+ @staticmethod
677
+ def _unpack(prediction: dict) -> tuple[str, float, float, float]:
678
+ """Extract prediction, confidence, and logit values from output dict."""
679
+ import torch
680
+
681
+ pred = prediction["prediction"]
682
+ confidence = float(prediction["confidence"])
683
+ logits = prediction["logits"]
684
+
685
+ # Handle Tensor or list
686
+ if isinstance(logits, torch.Tensor):
687
+ vals = logits.squeeze().tolist()
688
+ else:
689
+ vals = list(logits)
690
+
691
+ if isinstance(vals, float):
692
+ vals = [vals, vals]
693
+
694
+ return pred, confidence, round(vals[0], 4), round(vals[1], 4)
695
+
696
+ @staticmethod
697
+ def _split_generated(text: str, gradcam_context: Optional[str]) -> tuple[str, str]:
698
+ """
699
+ Split FLAN-T5 output into summary and detail.
700
+ Uses sentence boundary: first 2 sentences β†’ summary, rest β†’ detail.
701
+ """
702
+ import re
703
+ sentences = re.split(r'(?<=[.!?])\s+', text.strip())
704
+ sentences = [s.strip() for s in sentences if s.strip()]
705
+
706
+ if len(sentences) >= 3:
707
+ summary = " ".join(sentences[:2])
708
+ detail = " ".join(sentences[2:])
709
+ elif len(sentences) == 2:
710
+ summary = sentences[0]
711
+ detail = sentences[1]
712
+ else:
713
+ summary = text
714
+ detail = ""
715
+
716
+ if gradcam_context and gradcam_context not in detail:
717
+ detail += f" Spatial analysis: {gradcam_context}"
718
+
719
+ return summary, detail
720
+
721
+ @staticmethod
722
+ def _summarise_heatmap(heatmap) -> str:
723
+ """
724
+ Convert a (224, 224) Grad-CAM heatmap into a spatial text description.
725
+ Divides into 3Γ—3 grid, reports regions with activation above threshold.
726
+ """
727
+ import numpy as np
728
+
729
+ h, w = heatmap.shape
730
+ grid_h = h // 3
731
+ grid_w = w // 3
732
+ threshold = 0.6
733
+
734
+ region_names = [
735
+ ["top-left", "top-centre", "top-right"],
736
+ ["middle-left", "centre", "middle-right"],
737
+ ["bottom-left", "bottom-centre", "bottom-right"],
738
+ ]
739
+
740
+ hot_regions = []
741
+ for row in range(3):
742
+ for col in range(3):
743
+ patch = heatmap[
744
+ row * grid_h : (row + 1) * grid_h,
745
+ col * grid_w : (col + 1) * grid_w,
746
+ ]
747
+ if float(patch.mean()) > threshold:
748
+ hot_regions.append(
749
+ f"{region_names[row][col]} ({patch.mean():.2f})"
750
+ )
751
+
752
+ overall_mean = float(heatmap.mean())
753
+ overall_max = float(heatmap.max())
754
+
755
+ if hot_regions:
756
+ return (
757
+ f"High activation in: {', '.join(hot_regions)}. "
758
+ f"Mean={overall_mean:.3f}, peak={overall_max:.3f}. "
759
+ f"These regions most influenced the prediction."
760
+ )
761
+ return (
762
+ f"No dominant activation region detected. "
763
+ f"Mean={overall_mean:.3f}, peak={overall_max:.3f}. "
764
+ f"Prediction driven by diffuse low-level features."
765
+ )
766
+
767
+
768
+
769
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
770
+ # β•‘ CHAT ENGINE β€” human-like conversational responses via FLAN-T5 β•‘
771
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
772
+
773
+ class ChatEngine:
774
+ """
775
+ Human-like conversational response engine powered by FLAN-T5-large.
776
+
777
+ Uses carefully designed prompts to make FLAN-T5 respond naturally β€”
778
+ like a knowledgeable colleague β€” rather than producing stiff templated text.
779
+
780
+ Each response incorporates:
781
+ - Patient name, age, and medical history (if provided)
782
+ - Specific scan numbers (confidence, logits, decision margin)
783
+ - Grad-CAM spatial findings
784
+ - Audience-appropriate tone and vocabulary
785
+ - Conversation history for multi-turn context
786
+
787
+ Falls back to rich deterministic responses if FLAN-T5 is not loaded.
788
+ """
789
+
790
+ def __init__(self, llm: "FlanT5Engine | None" = None) -> None:
791
+ self.llm = llm
792
+
793
+ def respond(
794
+ self,
795
+ message: str,
796
+ audience: str,
797
+ prediction: str,
798
+ confidence: float,
799
+ benign_logit: float,
800
+ malignant_logit: float,
801
+ spatial_summary: str = "",
802
+ history: list = None,
803
+ patient: dict = None,
804
+ ) -> str:
805
+ """
806
+ Generate a human-like conversational response.
807
+
808
+ Parameters
809
+ ----------
810
+ message : the user's question
811
+ audience : clinician | researcher | patient
812
+ prediction : benign | malignant
813
+ confidence : float [0, 1]
814
+ benign_logit : raw logit for benign class
815
+ malignant_logit : raw logit for malignant class
816
+ spatial_summary : Grad-CAM text description
817
+ history : list of prior {"role", "content"} dicts
818
+ patient : dict with name, age, sex, medical_history, symptoms, previous_scans
819
+ """
820
+ patient = patient or {}
821
+ history = history or []
822
+ is_mal = prediction == "malignant"
823
+ pct = f"{confidence:.1%}"
824
+ margin = abs(malignant_logit - benign_logit)
825
+ name = patient.get("name", "")
826
+ age = patient.get("age", 0)
827
+ p_history = patient.get("medical_history", "")
828
+ symptoms = patient.get("symptoms", "")
829
+
830
+ # Try FLAN-T5 first
831
+ if self.llm is not None:
832
+ try:
833
+ prompt = self._build_prompt(
834
+ message, audience, prediction, confidence,
835
+ benign_logit, malignant_logit, spatial_summary,
836
+ history, patient, is_mal, pct, margin
837
+ )
838
+ response = self.llm.generate_chat(prompt)
839
+ if len(response.split()) >= 15:
840
+ return response
841
+ except Exception as e:
842
+ print(f"[ChatEngine] FLAN-T5 failed ({e}) β€” using fallback.")
843
+
844
+ # Fallback: rich deterministic responses
845
+ return self._fallback(
846
+ message, audience, prediction, confidence,
847
+ benign_logit, malignant_logit, spatial_summary,
848
+ is_mal, pct, margin, name, age, p_history, symptoms
849
+ )
850
+
851
+ # ── Prompt builder ────────────────────────────────────────────────────────
852
+
853
+ def _build_prompt(
854
+ self, message, audience, prediction, confidence,
855
+ b_logit, m_logit, cam, history, patient,
856
+ is_mal, pct, margin
857
+ ) -> str:
858
+ """Build a FLAN-T5 conversational prompt with full context."""
859
+
860
+ name = patient.get("name", "")
861
+ age = patient.get("age", 0)
862
+ sex = patient.get("sex", "")
863
+ hist = patient.get("medical_history", "")
864
+ symp = patient.get("symptoms", "")
865
+ scans = patient.get("previous_scans", "")
866
+
867
+ patient_ctx = ""
868
+ if name or age or hist or symp:
869
+ patient_ctx = f"""
870
+ Patient: {name or 'Anonymous'}{', age ' + str(age) if age else ''}{', ' + sex if sex else ''}.
871
+ {('Medical history: ' + hist) if hist else ''}
872
+ {('Symptoms: ' + symp) if symp else ''}
873
+ {('Previous scans: ' + scans) if scans else ''}
874
+ """.strip()
875
+
876
+ audience_style = {
877
+ "clinician": (
878
+ "a consultant radiologist. Use clinical terminology. "
879
+ "Be precise and collegial. Reference BI-RADS, logit margins, "
880
+ "and clinical decision context naturally."
881
+ ),
882
+ "researcher": (
883
+ "an ML researcher. Be technical. Reference softmax probabilities, "
884
+ "logit values, model architecture (DenseNet-121), training methodology "
885
+ "(OneCycleLR, Mixup, StainJitter), and calibration naturally."
886
+ ),
887
+ "patient": (
888
+ "a patient with no medical background. Be warm, empathetic, and clear. "
889
+ "Use plain English. No jargon. Acknowledge their feelings. "
890
+ "Be reassuring but honest. Address them by name if you know it."
891
+ ),
892
+ }.get(audience, "a medical professional")
893
+
894
+ history_ctx = ""
895
+ if history:
896
+ recent = history[-4:]
897
+ lines = [('User' if h.get('role')=='user' else 'Assistant')+': '+h.get('content','') for h in recent]
898
+ history_ctx = 'Previous exchanges: ' + ' | '.join(lines)
899
+
900
+ prompt = f"""You are a knowledgeable and empathetic AI medical assistant for the MedAI platform.
901
+ You are speaking to {audience_style}
902
+
903
+ Scan result:
904
+ - Classification: {prediction.upper()} ({pct} confidence)
905
+ - Logit scores: benign={b_logit:.4f}, malignant={m_logit:.4f} (margin={margin:.4f})
906
+ - Grad-CAM: {cam or 'Not available'}
907
+ - Model accuracy: 88.0%, sensitivity: 87.5%
908
+ {patient_ctx}
909
+ {history_ctx}
910
+
911
+ The person asks: "{message}"
912
+
913
+ Respond naturally and warmly in 2-4 sentences. Be specific β€” reference the actual numbers. Sound like a knowledgeable colleague, not a robot. End with a brief reminder that clinical confirmation is required.
914
+
915
+ Response:"""
916
+
917
+ return prompt.strip()
918
+
919
+ # ── Fallback: rich deterministic responses ────────────────────────────────
920
+
921
+ def _fallback(
922
+ self, message, audience, prediction, confidence,
923
+ b_logit, m_logit, cam, is_mal, pct, margin,
924
+ name, age, p_history, symptoms
925
+ ) -> str:
926
+ """Rich, human-sounding deterministic responses as fallback."""
927
+ msg = message.lower()
928
+ addr = f"{name.split()[0]}, " if name else ""
929
+ scan = f"{'malignant' if is_mal else 'benign'} at {pct} confidence"
930
+
931
+ if audience == "patient":
932
+ return self._patient_fallback(msg, is_mal, pct, margin, cam, addr, name, confidence, symptoms)
933
+ elif audience == "researcher":
934
+ return self._researcher_fallback(msg, is_mal, pct, b_logit, m_logit, margin, cam)
935
+ else:
936
+ return self._clinician_fallback(msg, is_mal, pct, b_logit, m_logit, margin, cam, confidence, p_history)
937
+
938
+ def _patient_fallback(self, msg, is_mal, pct, margin, cam, addr, name, conf, symptoms) -> str:
939
+ if any(k in msg for k in ["worry","worried","scared","serious","cancer","bad"]):
940
+ if is_mal:
941
+ return (
942
+ f"{addr}I completely understand why you might feel anxious right now β€” "
943
+ f"that's a very natural reaction. What I can tell you is that this AI "
944
+ f"flagged something that needs a closer look, and your doctor is the right "
945
+ f"person to interpret this alongside your full clinical picture. "
946
+ f"Many findings like this turn out to be benign on further investigation. "
947
+ f"Please don't make any decisions until you've spoken with your healthcare provider."
948
+ )
949
+ else:
950
+ return (
951
+ f"{addr}I can hear that this has been worrying for you. "
952
+ f"The good news is that the AI found no signs of abnormal tissue in this sample β€” "
953
+ f"that's a reassuring result. Of course, your doctor will want to confirm this "
954
+ f"as part of your overall care, but this is genuinely positive."
955
+ )
956
+ if any(k in msg for k in ["mean","understand","explain","what is","tell me"]):
957
+ if is_mal:
958
+ return (
959
+ f"{addr}think of the AI like a very experienced set of eyes that has studied "
960
+ f"thousands of tissue samples. It noticed patterns in this image β€” at {pct} confidence β€” "
961
+ f"that it has learned to associate with abnormal cells. "
962
+ f"That said, this is a screening tool, not a diagnosis. "
963
+ f"Your doctor will look at this result together with everything else they know about you."
964
+ )
965
+ else:
966
+ return (
967
+ f"{addr}the AI examined the patterns in this tissue sample and found that they "
968
+ f"look consistent with normal, healthy tissue β€” it's {pct} confident in that assessment. "
969
+ f"That's a really good sign. Your doctor will confirm this at your next appointment."
970
+ )
971
+ if any(k in msg for k in ["next","step","do","happen","biopsy","test"]):
972
+ if is_mal:
973
+ return (
974
+ f"{addr}the most important next step is to have a conversation with your doctor "
975
+ f"about this result as soon as possible. They may recommend additional imaging "
976
+ f"or a biopsy β€” which is a small, simple procedure to collect a tiny tissue sample "
977
+ f"for a laboratory to examine more closely. "
978
+ f"Please don't let anxiety about what might happen stop you from making that appointment."
979
+ )
980
+ else:
981
+ return (
982
+ f"{addr}with a reassuring result like this, your doctor will likely recommend "
983
+ f"continuing with your routine screening schedule. Do mention this result at your "
984
+ f"next appointment so it becomes part of your medical record. "
985
+ f"Is there anything else you'd like to understand about what this means?"
986
+ )
987
+ if any(k in msg for k in ["accurate","right","trust","sure","certain","reliable"]):
988
+ return (
989
+ f"{addr}that's a really important question to ask. The AI was correct on {pct} "
990
+ f"of test cases it hadn't seen before β€” which is good, but not perfect. "
991
+ f"No AI system is 100% accurate, which is exactly why your doctor always "
992
+ f"reviews the result before any clinical decision is made. "
993
+ f"Think of it as a very thorough first opinion."
994
+ )
995
+ if any(k in msg for k in ["heatmap","colour","color","red","highlighted","image","overlay"]):
996
+ return (
997
+ f"{addr}the coloured image you're seeing is called a Grad-CAM heatmap. "
998
+ f"The red and orange areas show where the AI was paying the most attention "
999
+ f"when it made its decision β€” those are the parts of the tissue it found "
1000
+ f"most significant. Blue areas were largely ignored. "
1001
+ + (f"In your scan, the AI was particularly focused on {cam.split('.')[0].lower()}." if cam else
1002
+ "Your doctor can use this to understand exactly what the AI was looking at.")
1003
+ )
1004
+ # Generic fallback
1005
+ return (
1006
+ f"{addr}I'm here to help you make sense of this result. "
1007
+ f"The AI classified this sample as {'potentially abnormal' if is_mal else 'normal-looking'} "
1008
+ f"at {pct} confidence. "
1009
+ f"You can ask me things like 'what does this mean?', 'should I be worried?', "
1010
+ f"or 'what happens next?' β€” and I'll do my best to explain clearly and honestly."
1011
+ )
1012
+
1013
+ def _clinician_fallback(self, msg, is_mal, pct, b_logit, m_logit, margin, cam, conf, p_history) -> str:
1014
+ birad = ("BI-RADS 4B (Suspicious)" if conf >= 0.75 else "BI-RADS 4A (Low suspicion)") if is_mal else "BI-RADS 2 (Benign)"
1015
+ boundary = ("clear decision boundary" if margin > 1.5 else "near the decision boundary β€” borderline case")
1016
+
1017
+ if any(k in msg for k in ["why","reason","basis","how","drove"]):
1018
+ return (
1019
+ f"The classifier scored this patch {'malignant' if is_mal else 'benign'} based on "
1020
+ f"DenseNet-121 feature activations β€” logits benign={b_logit:.4f}, malignant={m_logit:.4f}, "
1021
+ f"margin={margin:.4f} ({boundary}). "
1022
+ + (f"Grad-CAM identifies high activation in {cam.split('.')[0].lower()}, suggesting those "
1023
+ f"spatial regions drove the classification." if cam else
1024
+ f"Grad-CAM overlay is available in the heatmap tab for region-level review.") +
1025
+ f" Clinical correlation with morphological features is warranted."
1026
+ )
1027
+ if any(k in msg for k in ["birad","bi-rad","category","score","stage"]):
1028
+ return (
1029
+ f"Based on an AI confidence of {pct} and a logit margin of {margin:.3f}, "
1030
+ f"a suggested starting point is {birad}. "
1031
+ f"This is an AI-assisted recommendation only β€” final BI-RADS assignment "
1032
+ f"requires full clinical, imaging, and patient history correlation by the "
1033
+ f"responsible radiologist. "
1034
+ + (f"Relevant history: {p_history[:100]}..." if p_history else "")
1035
+ )
1036
+ if any(k in msg for k in ["biopsy","next","action","recommend","management"]):
1037
+ if is_mal:
1038
+ return (
1039
+ f"With a {pct} confidence malignant classification and a logit margin of {margin:.3f}, "
1040
+ f"{'tissue biopsy for histological confirmation is recommended' if conf >= 0.70 else 'short-interval follow-up imaging (6 months) may be appropriate given the borderline confidence'}. "
1041
+ f"Full clinical workup β€” including prior imaging comparison and patient history β€” "
1042
+ f"should precede any intervention decision."
1043
+ )
1044
+ else:
1045
+ return (
1046
+ f"With a {pct} confidence benign result and margin {margin:.3f}, "
1047
+ f"routine follow-up per standard screening protocol is appropriate. "
1048
+ f"If clinical suspicion remains high despite the AI result, "
1049
+ f"conventional workup should proceed independently of this output."
1050
+ )
1051
+ if any(k in msg for k in ["confident","confidence","reliable","calibrat"]):
1052
+ return (
1053
+ f"The softmax confidence of {pct} is uncalibrated β€” no temperature scaling "
1054
+ f"or isotonic regression was applied post-hoc. The underlying model achieved "
1055
+ f"87.5% sensitivity and 88.0% accuracy on 32,768 held-out PCam patches. "
1056
+ f"{'A margin of ' + str(round(margin,3)) + ' above 1.5 indicates strong separation from the decision boundary.' if margin > 1.5 else 'A margin of ' + str(round(margin,3)) + ' below 1.5 suggests caution β€” this is a borderline result.'}"
1057
+ )
1058
+ if any(k in msg for k in ["gradcam","grad-cam","heatmap","attention","region"]):
1059
+ return (
1060
+ f"Grad-CAM computed βˆ‚score_{('malignant' if is_mal else 'benign')}/βˆ‚A_k across "
1061
+ f"the norm5 feature layer (1024Γ—7Γ—7 spatial maps), globally average-pooled "
1062
+ f"to derive channel importance weights. "
1063
+ + (f"High-activation regions: {cam} β€” these locations contributed most to the "
1064
+ f"classification. The overlay is viewable in the Grad-CAM tab." if cam else
1065
+ "No dominant activation region detected β€” prediction driven by diffuse features.")
1066
+ )
1067
+ return (
1068
+ f"The model returned {'malignant' if is_mal else 'benign'} at {pct} confidence "
1069
+ f"(logits: b={b_logit:.4f}, m={m_logit:.4f}, margin={margin:.4f}). "
1070
+ f"I can elaborate on BI-RADS scoring, biopsy guidance, Grad-CAM interpretation, "
1071
+ f"confidence calibration, or model performance β€” what would be most useful?"
1072
+ )
1073
+
1074
+ def _researcher_fallback(self, msg, is_mal, pct, b_logit, m_logit, margin, cam) -> str:
1075
+ softmax_b = round(1 / (1 + 2.718 ** (m_logit - b_logit)), 4)
1076
+ softmax_m = round(1 - softmax_b, 4)
1077
+
1078
+ if any(k in msg for k in ["logit","score","raw","softmax","probability","output"]):
1079
+ return (
1080
+ f"Raw logits: [benign={b_logit:.6f}, malignant={m_logit:.6f}]. "
1081
+ f"After softmax: [P(benign)={softmax_b:.4f}, P(malignant)={softmax_m:.4f}]. "
1082
+ f"|Ξ”logit| = {margin:.6f} β€” "
1083
+ f"{'above the empirical 1.5 threshold for high-confidence separation' if margin > 1.5 else 'below 1.5, suggesting proximity to the decision boundary'}. "
1084
+ f"No temperature scaling or calibration applied post-hoc."
1085
+ )
1086
+ if any(k in msg for k in ["gradcam","grad-cam","gradient","activation","feature","saliency"]):
1087
+ return (
1088
+ f"Grad-CAM implementation: forward hook on model.features.norm5 (BΓ—1024Γ—7Γ—7). "
1089
+ f"Backward pass computes βˆ‚score_{{'malignant' if is_mal else 'benign'}}/βˆ‚A_k for each channel k. "
1090
+ f"Global average pooling of gradients gives weights Ξ±_k. "
1091
+ f"CAM = ReLU(Ξ£_k Ξ±_k Β· A_k), bilinearly upsampled 7Γ—7 β†’ 224Γ—224. "
1092
+ + (f"Spatial summary: {cam}." if cam else "No dominant activation detected.")
1093
+ )
1094
+ if any(k in msg for k in ["train","architecture","model","densenet","weight","epoch"]):
1095
+ return (
1096
+ f"DenseNet-121 (7,219,330 params) fine-tuned on 220,025 deduplicated PCam patches. "
1097
+ f"Training: OneCycleLR(max_lr=3e-3, pct_start=0.3), Mixup(Ξ±=0.4), "
1098
+ f"StainJitter(HED, strength=0.05), LabelSmoothing(0.1), "
1099
+ f"CrossEntropyLoss(pos_weight=1.469). "
1100
+ f"Checkpoint selection by val_sensitivity β€” best epoch 13/20, val_sens=0.903, "
1101
+ f"test_sens=0.875, test_acc=0.880."
1102
+ )
1103
+ if any(k in msg for k in ["dataset","pcam","camelyon","dedup","duplicate","balance"]):
1104
+ return (
1105
+ f"PatchCamelyon: 262,144 raw training patches β†’ 220,025 after MD5 deduplication "
1106
+ f"(42,119 removed, 83.9% retention). "
1107
+ f"The original dataset was artificially balanced by duplicating malignant patches β€” "
1108
+ f"post-dedup true distribution: benign=130,908, malignant=89,117 (pos_weight=1.469). "
1109
+ f"Deduplication was the single largest contributor to the +6.8pp sensitivity improvement."
1110
+ )
1111
+ if any(k in msg for k in ["calibrat","uncertain","temperature","reliability","ece"]):
1112
+ return (
1113
+ f"P({('malignant' if is_mal else 'benign')})={pct} is an uncalibrated softmax output. "
1114
+ f"No temperature scaling, Platt scaling, or isotonic regression was applied. "
1115
+ f"ECE was not computed on this checkpoint. "
1116
+ f"For reliable probability estimates, recommend fitting calibration on a held-out set "
1117
+ f"using sklearn.calibration.CalibratedClassifierCV or manual temperature scaling on logits."
1118
+ )
1119
+ return (
1120
+ f"Output: {('malignant' if is_mal else 'benign').upper()} | "
1121
+ f"logits=[{b_logit:.4f}, {m_logit:.4f}] | softmax=[{softmax_b:.4f}, {softmax_m:.4f}] | "
1122
+ f"|Ξ”logit|={margin:.4f}. "
1123
+ f"I can go deeper on logit analysis, Grad-CAM implementation, model architecture, "
1124
+ f"dataset statistics, or calibration. What would you like to explore?"
1125
+ )
explainability/mammogram_gradcam.py ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ explainability/mammogram_gradcam.py
3
+ ────────────────────────────────────
4
+ Grad-CAM for the EfficientNet-B4 mammogram models.
5
+
6
+ Why a separate class from gradcam.py?
7
+ ─────────────────────────────────────
8
+ The histopathology GradCAM in gradcam.py is hardwired for DenseNet-121:
9
+ β€’ it hooks model.features["norm5"] β€” EfficientNet has no norm5
10
+ β€’ it expects model(x) β†’ {"logits","probs"} dict
11
+ β€’ it preprocesses at 224Γ—224 β€” mammograms train at 384Γ—384
12
+
13
+ The mammogram ensemble members (model/mammogram_ensemble.py::_Model) are
14
+ EfficientNet-B4 with a .feat / .pool / .head structure and a bare-tensor
15
+ forward. This class targets that architecture directly.
16
+
17
+ It hooks the final EfficientNet feature block (model.feat), which outputs
18
+ (B, 1792, 12, 12) at 384Γ—384 input, computes gradient-weighted activations
19
+ for the target class, and upsamples to a 384Γ—384 heatmap + overlay.
20
+
21
+ Returns the same dict schema the API expects:
22
+ prediction, confidence, logits (Tensor[1,2]), heatmap (384Β²), overlay (PIL), cam_raw
23
+ """
24
+
25
+ from __future__ import annotations
26
+
27
+ from typing import Optional, Union
28
+
29
+ import numpy as np
30
+ import torch
31
+ import torch.nn as nn
32
+ import torch.nn.functional as F
33
+ from PIL import Image
34
+ from torchvision import transforms
35
+
36
+ LABEL_MAP = {0: "benign", 1: "malignant"}
37
+ MEAN = [0.485, 0.456, 0.406]
38
+ STD = [0.229, 0.224, 0.225]
39
+ INPUT_SIZE = 384 # mammogram models were trained at 384Γ—384
40
+
41
+
42
+ def _jet_colormap_vectorized(heatmap: np.ndarray) -> np.ndarray:
43
+ """(H,W) float [0,1] β†’ (H,W,3) uint8 RGB via jet colourmap. Vectorised."""
44
+ v = np.clip(heatmap, 0.0, 1.0)
45
+ r = np.clip(1.5 - np.abs(4 * v - 3), 0, 1)
46
+ g = np.clip(1.5 - np.abs(4 * v - 2), 0, 1)
47
+ b = np.clip(1.5 - np.abs(4 * v - 1), 0, 1)
48
+ return (np.stack([r, g, b], axis=-1) * 255).astype(np.uint8)
49
+
50
+
51
+ class MammogramGradCAM:
52
+ """
53
+ Grad-CAM for a single EfficientNet-B4 mammogram model.
54
+
55
+ Parameters
56
+ ----------
57
+ model : the _Model instance (or any module exposing `.feat`).
58
+ device : "cuda" | "mps" | "cpu". Auto-detected if None.
59
+ alpha : overlay blend weight. Default 0.5.
60
+ """
61
+
62
+ def __init__(
63
+ self,
64
+ model: nn.Module,
65
+ device: Optional[str] = None,
66
+ alpha: float = 0.5,
67
+ ) -> None:
68
+ self.device = self._resolve_device(device)
69
+ self.alpha = alpha
70
+ self.model = model.eval().to(self.device)
71
+
72
+ self.transform = transforms.Compose([
73
+ transforms.Resize((INPUT_SIZE, INPUT_SIZE)),
74
+ transforms.ToTensor(),
75
+ transforms.Normalize(MEAN, STD),
76
+ ])
77
+
78
+ self._activations: Optional[torch.Tensor] = None
79
+ self._gradients: Optional[torch.Tensor] = None
80
+
81
+ # Hook the final EfficientNet feature block.
82
+ # model.feat is the EfficientNet `features` Sequential; its output is
83
+ # the deepest spatial feature map before global pooling.
84
+ target = self._find_target_layer(model)
85
+ target.register_forward_hook(self._save_activations)
86
+ target.register_full_backward_hook(self._save_gradients)
87
+
88
+ @staticmethod
89
+ def _find_target_layer(model: nn.Module) -> nn.Module:
90
+ """Locate the conv feature module to hook."""
91
+ if hasattr(model, "feat"):
92
+ return model.feat # ensemble _Model
93
+ if hasattr(model, "features"):
94
+ return model.features # torchvision-style EfficientNet
95
+ raise AttributeError(
96
+ "Model has neither `.feat` nor `.features` β€” cannot attach Grad-CAM."
97
+ )
98
+
99
+ # ── Hook callbacks ──────────────────────────────────────────────────────
100
+ def _save_activations(self, module, inp, out) -> None:
101
+ self._activations = out.detach()
102
+
103
+ def _save_gradients(self, module, grad_in, grad_out) -> None:
104
+ self._gradients = grad_out[0].detach()
105
+
106
+ # ── Main API ────────────────────────────────────────────────────────────
107
+ def explain(
108
+ self,
109
+ image: Union[Image.Image, str],
110
+ class_idx: Optional[int] = None,
111
+ ) -> dict:
112
+ if isinstance(image, str):
113
+ image = Image.open(image).convert("RGB")
114
+ else:
115
+ image = image.convert("RGB")
116
+
117
+ tensor = self.transform(image).unsqueeze(0).to(self.device)
118
+ tensor.requires_grad_(True)
119
+
120
+ # Forward β€” _Model returns a bare (1,2) logits tensor
121
+ logits = self.model(tensor)
122
+ if isinstance(logits, dict): # tolerate dict-returning models
123
+ logits = logits["logits"]
124
+ probs = torch.softmax(logits, dim=1)
125
+
126
+ if class_idx is None:
127
+ class_idx = int(torch.argmax(probs, dim=1).item())
128
+ confidence = float(probs[0, class_idx].item())
129
+ prediction = LABEL_MAP[class_idx]
130
+
131
+ # Backward β€” gradients of the class score w.r.t. feature maps
132
+ self.model.zero_grad()
133
+ logits[0, class_idx].backward()
134
+
135
+ # Grad-CAM: channel weights = mean gradient over spatial dims
136
+ weights = self._gradients.mean(dim=(2, 3), keepdim=True) # (1,C,1,1)
137
+ cam = (weights * self._activations).sum(dim=1, keepdim=True)# (1,1,h,w)
138
+ cam = torch.relu(cam)
139
+
140
+ cam_raw = cam[0, 0].cpu().numpy()
141
+ mn, mx = cam_raw.min(), cam_raw.max()
142
+ cam_norm = (cam_raw - mn) / (mx - mn) if mx > mn else np.zeros_like(cam_raw)
143
+
144
+ heatmap = self._upsample(cam_norm, (INPUT_SIZE, INPUT_SIZE))
145
+ overlay = self._build_overlay(image.resize((INPUT_SIZE, INPUT_SIZE)), heatmap)
146
+
147
+ return {
148
+ "prediction": prediction,
149
+ "confidence": round(confidence, 6),
150
+ "logits": logits.detach().cpu(),
151
+ "heatmap": heatmap,
152
+ "overlay": overlay,
153
+ "cam_raw": cam_raw,
154
+ }
155
+
156
+ # ── Helpers ─────────────────────────────────────────────────────────────
157
+ @staticmethod
158
+ def _upsample(cam: np.ndarray, size: tuple[int, int]) -> np.ndarray:
159
+ t = torch.from_numpy(cam).float()[None, None]
160
+ up = F.interpolate(t, size=size, mode="bilinear", align_corners=False)
161
+ return up[0, 0].numpy()
162
+
163
+ def _build_overlay(self, original: Image.Image, heatmap: np.ndarray) -> Image.Image:
164
+ jet = Image.fromarray(_jet_colormap_vectorized(heatmap), mode="RGB")
165
+ return Image.blend(original.convert("RGB"), jet, alpha=self.alpha)
166
+
167
+ @staticmethod
168
+ def _resolve_device(device: Optional[str]) -> torch.device:
169
+ if device is not None:
170
+ return torch.device(device)
171
+ if torch.cuda.is_available():
172
+ return torch.device("cuda")
173
+ if torch.backends.mps.is_available():
174
+ return torch.device("mps")
175
+ return torch.device("cpu")
frontend/index.html ADDED
@@ -0,0 +1,1073 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8"/>
5
+ <meta name="viewport" content="width=device-width,initial-scale=1.0"/>
6
+ <title>MedAI β€” Breast Cancer Analysis Platform</title>
7
+ <link rel="preconnect" href="https://fonts.googleapis.com"/>
8
+ <link href="https://fonts.googleapis.com/css2?family=Syne:wght@400;600;700;800&family=Sora:wght@300;400;500;600&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet"/>
9
+ <style>
10
+ *,*::before,*::after{box-sizing:border-box;margin:0;padding:0}
11
+ :root{
12
+ --bg:#06070A;--bg2:#0E1016;--bg3:#161922;
13
+ --border:rgba(255,255,255,.08);--border2:rgba(255,255,255,.14);
14
+ --text:#e8f0ff;--muted:rgba(232,240,255,.66);
15
+ --teal:#2E8BF5;--teal-dim:rgba(46,139,245,.15);--teal-lt:#A8CBFF;
16
+ --red:#E24B4A;--red-dim:rgba(226,75,74,.15);
17
+ --radius:10px;
18
+ }
19
+ body{font-family:'Sora',sans-serif;background:var(--bg);color:var(--text);min-height:100vh;font-size:14px;line-height:1.6}
20
+ button{cursor:pointer;font-family:'Sora',sans-serif}
21
+ .nav{display:flex;align-items:center;justify-content:space-between;padding:14px 32px;border-bottom:.5px solid var(--border);background:var(--bg);position:sticky;top:0;z-index:100}
22
+ .logo{width:34px;height:34px;border-radius:9px;background:var(--teal);display:flex;align-items:center;justify-content:center;font-family:'Syne',sans-serif;font-weight:800;font-size:16px;color:#EAF2FF}
23
+ .nav-title{font-family:'Syne',sans-serif;font-weight:800;font-size:17px}
24
+ .nav-badge{font-size:10px;padding:2px 8px;border-radius:4px;background:var(--teal-dim);color:var(--teal-lt);font-weight:600}
25
+ .nbtn{padding:7px 16px;border-radius:7px;border:.5px solid transparent;background:transparent;color:var(--muted);font-size:13px;transition:all .15s}
26
+ .nbtn:hover{background:var(--bg2);color:var(--text)}
27
+ .nbtn.active{background:var(--teal-dim);color:var(--teal-lt);border-color:rgba(46,139,245,.3)}
28
+ .btn-p{padding:9px 20px;border-radius:8px;border:none;font-size:13px;font-weight:600;background:var(--teal);color:#EAF2FF;transition:background .15s}
29
+ .btn-p:hover{background:#1C72DE}
30
+ .btn-p:disabled{opacity:.5;cursor:not-allowed}
31
+ .btn-o{padding:9px 20px;border-radius:8px;border:.5px solid var(--border2);font-size:13px;background:transparent;color:var(--text);transition:background .15s}
32
+ .btn-o:hover{background:var(--bg2)}
33
+ .page{padding:36px 32px;max-width:1200px;margin:0 auto}
34
+ .hidden{display:none!important}
35
+ .card{background:var(--bg2);border:.5px solid var(--border);border-radius:var(--radius);padding:20px}
36
+ .card-sm{background:var(--bg2);border:.5px solid var(--border);border-radius:8px;padding:14px}
37
+ .slabel{font-size:11px;font-weight:600;letter-spacing:.12em;text-transform:uppercase;color:var(--teal);margin-bottom:8px}
38
+ .h1{font-family:'Syne',sans-serif;font-weight:800;font-size:40px;line-height:1.1}
39
+ .h2{font-family:'Syne',sans-serif;font-weight:700;font-size:26px}
40
+ .mono{font-family:'JetBrains Mono',monospace}
41
+ .f{display:flex}.fc{display:flex;flex-direction:column}
42
+ .ac{align-items:center}.jb{justify-content:space-between}
43
+ .g6{gap:6px}.g8{gap:8px}.g10{gap:10px}.g12{gap:12px}.g16{gap:16px}.g24{gap:24px}
44
+ .mt8{margin-top:8px}.mt12{margin-top:12px}.mt16{margin-top:16px}.mt20{margin-top:20px}.mt24{margin-top:24px}
45
+ .g2{display:grid;grid-template-columns:1fr 1fr;gap:16px}
46
+ .g4{display:grid;grid-template-columns:repeat(4,1fr);gap:12px}
47
+ .bdanger{display:inline-flex;align-items:center;gap:6px;padding:5px 12px;border-radius:7px;background:var(--red-dim);color:#F09595;font-size:13px;font-weight:600;border:.5px solid rgba(226,75,74,.3)}
48
+ .bsafe{display:inline-flex;align-items:center;gap:6px;padding:5px 12px;border-radius:7px;background:var(--teal-dim);color:var(--teal-lt);font-size:13px;font-weight:600;border:.5px solid rgba(46,139,245,.3)}
49
+ .bteal{display:inline-flex;align-items:center;gap:6px;padding:4px 12px;border-radius:20px;background:var(--teal-dim);color:var(--teal-lt);font-size:12px;font-weight:500}
50
+ .dlive{width:7px;height:7px;border-radius:50%;background:var(--teal);animation:pulse 1.5s ease infinite;flex-shrink:0}
51
+ .upload-zone{border:1.5px dashed var(--border2);border-radius:var(--radius);padding:40px 24px;text-align:center;position:relative;transition:all .2s}
52
+ .upload-zone:hover{border-color:var(--teal);background:var(--teal-dim)}
53
+ .upload-zone input[type=file]{position:absolute;inset:0;opacity:0;width:100%;cursor:pointer}
54
+ .tab-bar{display:flex;gap:4px;border-bottom:.5px solid var(--border);margin-bottom:20px}
55
+ .tab{padding:8px 16px;font-size:13px;background:none;border:none;border-bottom:2px solid transparent;color:var(--muted);transition:all .15s}
56
+ .tab:hover{color:var(--text)}
57
+ .tab.active{color:var(--teal-lt);border-bottom-color:var(--teal)}
58
+ .cbar{height:6px;border-radius:3px;background:var(--bg3);overflow:hidden}
59
+ .cfill{height:100%;border-radius:3px;transition:width .6s ease}
60
+ .mrow{display:flex;justify-content:space-between;align-items:center;padding:9px 0;border-bottom:.5px solid var(--border)}
61
+ .mrow:last-child{border-bottom:none}
62
+ .simg{width:100%;aspect-ratio:1;border-radius:var(--radius);object-fit:cover;background:var(--bg3);border:.5px solid var(--border);display:block}
63
+ .scan-tile{width:100%;aspect-ratio:1;object-fit:cover;border-radius:var(--radius);border:.5px solid var(--border);background:var(--bg3);display:block}
64
+ .scan-ph{width:100%;aspect-ratio:1;border-radius:var(--radius);border:.5px solid var(--border);background:var(--bg3);display:none;flex-direction:column;align-items:center;justify-content:center;gap:6px;color:var(--muted);font-size:12px}
65
+ .scan-cap{font-size:10px;color:var(--muted);margin-top:6px;text-align:center}
66
+ .sph{width:100%;aspect-ratio:1;border-radius:var(--radius);background:var(--bg3);border:.5px solid var(--border);display:flex;align-items:center;justify-content:center;flex-direction:column;gap:8px;color:var(--muted);font-size:13px}
67
+ .aud-sel{background:var(--bg3);border:.5px solid var(--border2);border-radius:7px;padding:7px 12px;color:var(--text);font-family:'Sora',sans-serif;font-size:13px;outline:none}
68
+ .aud-sel:focus{border-color:var(--teal)}
69
+ .chat-panel{border:.5px solid var(--border);border-radius:var(--radius);overflow:hidden;margin-top:24px}
70
+ .chat-head{padding:12px 18px;border-bottom:.5px solid var(--border);display:flex;align-items:center;justify-content:space-between;background:var(--bg2)}
71
+ .chat-body{height:280px;overflow-y:auto;padding:14px;display:flex;flex-direction:column;gap:10px}
72
+ .mai{align-self:flex-start;max-width:85%;background:var(--bg2);border:.5px solid var(--border);border-radius:10px 10px 10px 3px;padding:10px 14px;font-size:12.5px;line-height:1.65;animation:fi .25s ease;white-space:pre-wrap}
73
+ .muser{align-self:flex-end;max-width:85%;background:var(--teal);color:#EAF2FF;border-radius:10px 10px 3px 10px;padding:10px 14px;font-size:12.5px;line-height:1.65;animation:fi .25s ease}
74
+ .mai.md{white-space:normal}
75
+ .mai.md p{margin:0 0 8px}
76
+ .mai.md p:last-child{margin-bottom:0}
77
+ .mai.md h3,.mai.md h4,.mai.md h5{font-family:'Syne',sans-serif;margin:11px 0 6px;line-height:1.3;color:var(--text)}
78
+ .mai.md h3{font-size:15px}.mai.md h4{font-size:13.5px}.mai.md h5{font-size:12.5px}
79
+ .mai.md ul,.mai.md ol{margin:4px 0 8px;padding-left:18px}
80
+ .mai.md li{margin:2px 0}
81
+ .mai.md code{background:var(--bg3);padding:1px 5px;border-radius:4px;font-family:'JetBrains Mono',monospace;font-size:11.5px}
82
+ .mai.md pre{background:var(--bg3);border:.5px solid var(--border);border-radius:8px;padding:10px;overflow-x:auto;margin:8px 0}
83
+ .mai.md pre code{background:none;padding:0}
84
+ .mai.md strong{color:var(--text);font-weight:600}
85
+ .chat-foot{padding:10px 14px;border-top:.5px solid var(--border);display:flex;gap:8px;background:var(--bg2)}
86
+ .cinp{flex:1;padding:8px 12px;border-radius:7px;border:.5px solid var(--border2);background:var(--bg3);color:var(--text);font-size:12.5px;font-family:'Sora',sans-serif;outline:none}
87
+ .cinp:focus{border-color:var(--teal)}
88
+ .chip-row{padding:8px 14px;border-top:.5px solid var(--border);display:flex;gap:6px;flex-wrap:wrap;background:var(--bg2)}
89
+ .chip{padding:5px 10px;border-radius:16px;border:.5px solid var(--border2);font-size:11px;background:transparent;color:var(--muted);font-family:'Sora',sans-serif;transition:all .15s;white-space:nowrap}
90
+ .chip:hover{border-color:var(--teal);color:var(--teal-lt);background:var(--teal-dim)}
91
+ .td{display:inline-block;width:6px;height:6px;border-radius:50%;background:var(--teal);margin:0 2px;animation:pulse 1s ease infinite}
92
+ .td:nth-child(2){animation-delay:.2s}.td:nth-child(3){animation-delay:.4s}
93
+ .fcard{background:var(--bg2);border:.5px solid var(--border);border-radius:var(--radius);padding:20px;transition:border-color .2s}
94
+ .fcard:hover{border-color:var(--teal)}
95
+ .ficon{width:40px;height:40px;border-radius:10px;background:var(--teal-dim);display:flex;align-items:center;justify-content:center;margin-bottom:12px;font-size:18px}
96
+ .svgi{width:20px;height:20px;stroke:var(--teal-lt);fill:none;stroke-width:1.6;stroke-linecap:round;stroke-linejoin:round}
97
+ .site-footer{max-width:1200px;margin:24px auto 0;padding:0 32px 40px}
98
+ .footer-disc{background:var(--red-dim);color:#F09595;border:.5px solid rgba(226,75,74,.25);border-radius:8px;padding:11px 16px;font-size:12.5px;font-weight:600;text-align:center;margin-bottom:22px}
99
+ .footer-row{display:flex;justify-content:space-between;align-items:center;gap:20px;flex-wrap:wrap;border-top:.5px solid var(--border);padding-top:22px}
100
+ .site-footer a{color:var(--teal-lt);text-decoration:none}
101
+ .site-footer a:hover{text-decoration:underline}
102
+ .result-banner{display:flex;align-items:center;justify-content:space-between;padding:16px 18px;border-radius:var(--radius);border:.5px solid var(--border2)}
103
+ .result-banner.benign{background:var(--teal-dim);border-color:rgba(46,139,245,.4)}
104
+ .result-banner.suspicious{background:var(--red-dim);border-color:rgba(226,75,74,.4)}
105
+ .rb-verdict{display:flex;align-items:center;gap:9px;font-family:'Syne',sans-serif;font-weight:800;font-size:22px;letter-spacing:.01em}
106
+ .result-banner.benign .rb-verdict{color:var(--teal-lt)}
107
+ .result-banner.suspicious .rb-verdict{color:#F4A6A6}
108
+ .rb-conf{text-align:right;line-height:1.1}
109
+ .rb-conf .rb-num{font-family:'JetBrains Mono',monospace;font-size:23px;font-weight:600}
110
+ .rb-conf .rb-lbl{font-size:10px;text-transform:uppercase;letter-spacing:.14em;color:var(--muted);margin-top:2px}
111
+ .model-chip{display:inline-flex;align-items:center;gap:8px;font-size:11px;color:var(--muted);font-family:'JetBrains Mono',monospace;background:var(--bg3);padding:6px 11px;border-radius:6px;border:.5px solid var(--border)}
112
+ .adv{border-top:.5px solid var(--border);padding-top:10px}
113
+ .adv summary{cursor:pointer;font-size:11px;color:var(--muted);letter-spacing:.04em;list-style:none;user-select:none;display:flex;align-items:center;gap:6px}
114
+ .adv summary::-webkit-details-marker{display:none}
115
+ .adv summary::before{content:'β–Έ';transition:transform .15s}
116
+ .adv[open] summary::before{transform:rotate(90deg)}
117
+ .adv summary:hover{color:var(--text)}
118
+ .modbar{display:inline-flex;gap:4px;background:var(--bg2);border:.5px solid var(--border);border-radius:9px;padding:4px;margin-bottom:24px}
119
+ .modbtn{padding:7px 18px;border-radius:6px;border:none;background:transparent;color:var(--muted);font-size:13px;font-weight:500;font-family:'Sora',sans-serif;cursor:pointer;transition:all .15s}
120
+ .modbtn:hover{color:var(--text)}
121
+ .modbtn.active{background:var(--teal-dim);color:var(--teal-lt)}
122
+ .sample-row{display:flex;gap:12px;margin-top:10px;flex-wrap:wrap}
123
+ .sample{cursor:pointer;border:.5px solid var(--border);border-radius:8px;padding:8px;background:var(--bg2);transition:border-color .2s;text-align:center;width:106px}
124
+ .sample:hover{border-color:var(--teal)}
125
+ .sample img{width:90px;height:90px;object-fit:cover;border-radius:6px;display:block;background:var(--bg3)}
126
+ .sample span{display:block;font-size:11px;color:var(--muted);margin-top:6px}
127
+ .api-bar{background:var(--bg3);border:.5px solid var(--border);border-radius:8px;padding:12px 16px;font-size:12px;display:flex;align-items:center;gap:10px;margin-bottom:24px}
128
+ .api-inp{background:transparent;border:none;outline:none;color:var(--teal-lt);font-family:'JetBrains Mono',monospace;font-size:12px;flex:1}
129
+ .sn{font-family:'Syne',sans-serif;font-weight:800;font-size:28px;color:var(--teal)}
130
+ .stepn{width:32px;height:32px;border-radius:50%;background:var(--teal);display:flex;align-items:center;justify-content:center;font-family:'Syne',sans-serif;font-weight:700;font-size:13px;color:#EAF2FF;flex-shrink:0}
131
+ .pform{background:var(--bg2);border:.5px solid var(--border);border-radius:var(--radius);padding:16px;margin-bottom:16px}
132
+ .pform-grid{display:grid;grid-template-columns:1fr 1fr;gap:10px}
133
+ .pinp{width:100%;padding:8px 10px;border-radius:7px;border:.5px solid var(--border2);background:var(--bg3);color:var(--text);font-size:12px;font-family:'Sora',sans-serif;outline:none}
134
+ .pinp:focus{border-color:var(--teal)}
135
+ .plabel{font-size:11px;color:var(--muted);margin-bottom:4px;display:block}
136
+ .ptextarea{width:100%;padding:8px 10px;border-radius:7px;border:.5px solid var(--border2);background:var(--bg3);color:var(--text);font-size:12px;font-family:'Sora',sans-serif;outline:none;resize:vertical;min-height:60px}
137
+ .ptextarea:focus{border-color:var(--teal)}
138
+ .ptoggle{display:flex;align-items:center;gap:8px;cursor:pointer;font-size:12px;color:var(--muted);margin-bottom:12px;user-select:none}
139
+ .ptoggle:hover{color:var(--text)}
140
+ .ebox{font-size:12px;color:#F09595;background:var(--red-dim);border:.5px solid rgba(226,75,74,.3);border-radius:7px;padding:10px 14px;margin-top:8px}
141
+ @keyframes pulse{0%,100%{opacity:1}50%{opacity:.4}}
142
+ @keyframes fi{from{opacity:0;transform:translateY(5px)}to{opacity:1;transform:translateY(0)}}
143
+ @keyframes spin{to{transform:rotate(360deg)}}
144
+ </style>
145
+ </head>
146
+ <body>
147
+
148
+ <nav class="nav">
149
+ <div class="f ac g10">
150
+ <div class="logo">M</div>
151
+ <span class="nav-title">MedAI</span>
152
+ <span class="nav-badge">v2.0</span>
153
+ </div>
154
+ <div class="f g6">
155
+ <button class="nbtn active" id="nb-platform" onclick="show('platform')">Platform</button>
156
+ <button class="nbtn" id="nb-dashboard" onclick="show('dashboard')">Dashboard</button>
157
+ <button class="nbtn" id="nb-assistant" onclick="show('assistant')">AI Assistant</button>
158
+ <button class="nbtn" id="nb-research" onclick="show('research')">Research</button>
159
+ </div>
160
+ <div class="f g8 ac">
161
+ <button class="btn-o" style="font-size:12px;display:flex;align-items:center;gap:7px" onclick="checkHealth()">
162
+ <span id="apidot" style="width:7px;height:7px;border-radius:50%;background:#555;display:inline-block"></span><span id="apistatus">API status</span>
163
+ </button>
164
+ <button class="btn-p" onclick="show('dashboard')">Open dashboard β†’</button>
165
+ </div>
166
+ </nav>
167
+
168
+ <!-- PLATFORM -->
169
+ <div id="v-platform" class="page">
170
+ <div style="display:grid;grid-template-columns:1fr 1fr;gap:48px;align-items:center;padding-bottom:48px;border-bottom:.5px solid var(--border)">
171
+ <div>
172
+ <div class="bteal" style="margin-bottom:18px">Histopathology Β· Mammography Β· Explainable AI</div>
173
+ <h1 class="h1">AI-powered breast cancer intelligence platform</h1>
174
+ <p style="color:var(--muted);margin-top:14px;font-size:15px;max-width:480px">Research-grade breast cancer detection across two modalities β€” histopathology and mammography β€” with explainable AI. Upload a tissue patch or a mammogram and receive a prediction, a spatial heatmap, and a natural-language report in seconds.</p>
175
+ <div class="f g12 mt24">
176
+ <button class="btn-p" style="font-size:14px;padding:11px 24px" onclick="show('dashboard')">Open dashboard β†’</button>
177
+ <button class="btn-o" style="font-size:14px;padding:11px 24px" onclick="show('assistant')">Try AI assistant</button>
178
+ </div>
179
+ <div class="f g24 mt24" style="flex-wrap:wrap">
180
+ <div>
181
+ <div class="slabel" style="margin-bottom:8px">Histopathology module</div>
182
+ <div class="f g24">
183
+ <div><div class="sn">88.0%</div><div style="font-size:12px;color:var(--muted);margin-top:2px">Test accuracy</div></div>
184
+ <div><div class="sn">87.5%</div><div style="font-size:12px;color:var(--muted);margin-top:2px">Sensitivity</div></div>
185
+ </div>
186
+ </div>
187
+ <div>
188
+ <div class="slabel" style="margin-bottom:8px">Mammography module</div>
189
+ <div class="f g24">
190
+ <div><div class="sn">0.8443</div><div style="font-size:12px;color:var(--muted);margin-top:2px">Patient-level AUC</div></div>
191
+ </div>
192
+ </div>
193
+ </div>
194
+ </div>
195
+ <div>
196
+ <div class="mono" style="font-size:10px;color:var(--muted);margin-bottom:6px">SCAN VIEWER</div>
197
+ <div style="display:grid;grid-template-columns:1fr 1fr;gap:10px">
198
+ <div>
199
+ <img src="samples/histo-tumor.png" alt="Histopathology patch" class="scan-tile" onerror="this.style.display='none';this.nextElementSibling.style.display='flex'"/>
200
+ <div class="scan-ph"><svg class="svgi" style="width:30px;height:30px" viewBox="0 0 24 24"><circle cx="8" cy="8" r="2.4"/><circle cx="16" cy="8" r="2.4"/><circle cx="8" cy="16" r="2.4"/><circle cx="16" cy="16" r="2.4"/></svg><span>Histopathology</span></div>
201
+ <div class="scan-cap">Histopathology Β· H&amp;E patch</div>
202
+ </div>
203
+ <div>
204
+ <img src="samples/mammo-cancer.png" alt="Mammogram" class="scan-tile" onerror="this.style.display='none';this.nextElementSibling.style.display='flex'"/>
205
+ <div class="scan-ph"><svg class="svgi" style="width:30px;height:30px" viewBox="0 0 24 24"><path d="M4 20c0-7 4-12 8-12s8 5 8 12"/><circle cx="14" cy="13" r="2"/></svg><span>Mammography</span></div>
206
+ <div class="scan-cap">Mammography Β· DICOM view</div>
207
+ </div>
208
+ </div>
209
+ <div class="mono" style="font-size:10px;color:var(--muted);margin:12px 0 6px">SAMPLE RESULT β€” ILLUSTRATIVE</div>
210
+ <div class="g2">
211
+ <div class="card-sm"><div style="font-size:10px;color:var(--muted);margin-bottom:4px" class="mono">PREDICTION</div><div class="bdanger">⚠ MALIGNANT</div></div>
212
+ <div class="card-sm"><div style="font-size:10px;color:var(--muted);margin-bottom:4px" class="mono">CONFIDENCE</div><div style="font-family:'Syne',sans-serif;font-weight:800;font-size:22px;color:var(--red)">0.94</div></div>
213
+ </div>
214
+ </div>
215
+ </div>
216
+ <div style="padding:48px 0;border-bottom:.5px solid var(--border)">
217
+ <div class="slabel">Platform capabilities</div>
218
+ <h2 class="h2" style="margin-bottom:28px">Two detection modules, one explainable platform</h2>
219
+
220
+ <div class="slabel" style="margin-bottom:12px;color:var(--teal-lt)">Histopathology module</div>
221
+ <div style="display:grid;grid-template-columns:repeat(3,1fr);gap:14px">
222
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M12 2 2 7l10 5 10-5z"/><path d="m2 17 10 5 10-5"/><path d="m2 12 10 5 10-5"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">DenseNet-121</div><div style="font-size:12px;color:var(--muted);line-height:1.5">~7.2M-parameter CNN fine-tuned on 220K deduplicated PCam H&amp;E tissue patches.</div></div>
223
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M12 2.5S5 9 5 14a7 7 0 0 0 14 0c0-5-7-11.5-7-11.5z"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">StainJitter (H&amp;E)</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Stain augmentation in HED colour space, reducing lab-specific staining overfitting.</div></div>
224
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><circle cx="12" cy="12" r="9"/><circle cx="12" cy="12" r="5"/><circle cx="12" cy="12" r="1"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">Sensitivity-first</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Checkpoints selected by validation sensitivity β€” tuned to catch malignant patches.</div></div>
225
+ </div>
226
+
227
+ <div class="slabel" style="margin:24px 0 12px;color:var(--teal-lt)">Mammography module</div>
228
+ <div style="display:grid;grid-template-columns:repeat(3,1fr);gap:14px">
229
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><rect x="9" y="2" width="6" height="6" rx="1"/><rect x="2" y="16" width="6" height="6" rx="1"/><rect x="16" y="16" width="6" height="6" rx="1"/><path d="M12 8v4M5 16v-2h14v2"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">EfficientNet-B4 ensemble</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Three EfficientNet-B4 models (seeds 42 / 123 / 999) trained on RSNA data, averaged at inference.</div></div>
230
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><rect x="3" y="3" width="18" height="18" rx="2"/><circle cx="9" cy="9" r="2"/><path d="m21 15-3.6-3.6a2 2 0 0 0-2.8 0L6 21"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">DICOM + VOI LUT</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Native DICOM loading with VOI-LUT windowing. Also accepts PNG, JPG and TIFF.</div></div>
231
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><rect x="8" y="2" width="8" height="4" rx="1"/><path d="M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2"/><path d="m9 13 2 2 4-4"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">BI-RADS scoring</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Suggested BI-RADS category (2–5) from malignancy probability. Radiologist confirmation required.</div></div>
232
+ </div>
233
+
234
+ <div class="slabel" style="margin:24px 0 12px;color:var(--teal-lt)">Shared platform layers β€” both modules</div>
235
+ <div style="display:grid;grid-template-columns:repeat(4,1fr);gap:14px">
236
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M8.5 14.5A2.5 2.5 0 0 0 11 12c0-1.38-.5-2-1-3-1.07-2.14-.22-4.05 2-6 .5 2.5 2 4.9 4 6.5 2 1.6 3 3.5 3 5.5a7 7 0 1 1-14 0c0-1.15.43-2.29 1-3a2.5 2.5 0 0 0 2.5 2.5z"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">Grad-CAM heatmaps</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Spatial attention maps showing which regions drove each prediction.</div></div>
237
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><path d="M14 2v6h6M16 13H8M16 17H8M10 9H8"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">LLM explanation</div><div style="font-size:12px;color:var(--muted);line-height:1.5">FLAN-T5 / BioMedLM / Llama 3.2 reports in clinician, researcher or patient modes.</div></div>
238
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M21 11.5a8.38 8.38 0 0 1-.9 3.8 8.5 8.5 0 0 1-7.6 4.7 8.38 8.38 0 0 1-3.8-.9L3 21l1.9-5.7a8.38 8.38 0 0 1-.9-3.8 8.5 8.5 0 0 1 4.7-7.6 8.38 8.38 0 0 1 3.8-.9h.5a8.48 8.48 0 0 1 8 8z"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">Chat assistant</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Ask follow-up questions about any result, in clinician, researcher or patient mode.</div></div>
239
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="m12 14 4-4"/><path d="M3.34 19a10 10 0 1 1 17.32 0"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">Confidence scores</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Calibrated probability scores with raw logits and decision margin.</div></div>
240
+ </div>
241
+ </div>
242
+ <div style="padding:48px 0">
243
+ <div class="slabel">How it works</div>
244
+ <h2 class="h2" style="margin-bottom:28px">From scan to report in five steps</h2>
245
+ <div class="fc g16" style="max-width:600px">
246
+ <div class="f ac g16"><div class="stepn">1</div><div><div style="font-weight:600;font-size:14px;margin-bottom:3px">Upload a scan</div><div style="font-size:12px;color:var(--muted)">Histology patch (PNG / JPG / TIFF) or mammogram (DICOM / PNG) β€” drag and drop onto the dashboard</div></div></div>
247
+ <div class="f ac g16"><div class="stepn">2</div><div><div style="font-weight:600;font-size:14px;margin-bottom:3px">AI detection</div><div style="font-size:12px;color:var(--muted)">Routed by modality β€” DenseNet-121 for histology patches, the EfficientNet-B4 ensemble for mammograms</div></div></div>
248
+ <div class="f ac g16"><div class="stepn">3</div><div><div style="font-weight:600;font-size:14px;margin-bottom:3px">Grad-CAM overlay</div><div style="font-size:12px;color:var(--muted)">Spatial heatmap generated showing the regions that drove the prediction</div></div></div>
249
+ <div class="f ac g16"><div class="stepn">4</div><div><div style="font-weight:600;font-size:14px;margin-bottom:3px">LLM explanation</div><div style="font-size:12px;color:var(--muted)">Audience-tailored natural language report</div></div></div>
250
+ <div class="f ac g16"><div class="stepn">5</div><div><div style="font-weight:600;font-size:14px;margin-bottom:3px">Chat and review</div><div style="font-size:12px;color:var(--muted)">Ask follow-up questions in the inline chat assistant</div></div></div>
251
+ </div>
252
+ </div>
253
+ </div>
254
+
255
+ <!-- DASHBOARD -->
256
+ <div id="v-dashboard" class="page hidden">
257
+ <div class="f ac jb" style="margin-bottom:24px">
258
+ <div><h2 class="h2">Analysis dashboard</h2><p id="dash-sub" style="color:var(--muted);font-size:13px;margin-top:3px">Upload a histopathology patch to run the full pipeline</p></div>
259
+ <div class="f g8 ac">
260
+ <span style="font-size:12px;color:var(--muted)">Audience:</span>
261
+ <select class="aud-sel" id="audience">
262
+ <option value="clinician">Clinician</option>
263
+ <option value="researcher">Researcher</option>
264
+ <option value="patient">Patient</option>
265
+ </select>
266
+ </div>
267
+ </div>
268
+
269
+ <div class="modbar">
270
+ <button class="modbtn active" id="mod-histo" onclick="switchModality('histo')">Histopathology</button>
271
+ <button class="modbtn" id="mod-mammo" onclick="switchModality('mammo')">Mammogram</button>
272
+ </div>
273
+ <div id="dash-histo">
274
+ <div style="display:grid;grid-template-columns:1fr 1fr;gap:24px;align-items:start">
275
+ <div>
276
+ <div class="tab-bar">
277
+ <button class="tab active" onclick="switchTab('upload',this)">Upload scan</button>
278
+ <button class="tab" onclick="switchTab('overlay',this)">Grad-CAM overlay</button>
279
+ <button class="tab" onclick="switchTab('raw',this)">Raw heatmap</button>
280
+ </div>
281
+ <!-- Patient record (collapsible) -->
282
+ <div id="pform-wrap">
283
+ <div class="ptoggle" onclick="togglePatient()">
284
+ <span id="ptoggle-icon">β–Ά</span>
285
+ <span>Patient record <span style="font-size:10px;color:var(--teal);margin-left:4px">(optional β€” improves chat responses)</span></span>
286
+ </div>
287
+ <div id="pform-body" class="pform hidden">
288
+ <div class="pform-grid">
289
+ <div><label class="plabel">Full name</label><input class="pinp" id="p-name" placeholder="e.g. Amara Nwosu"/></div>
290
+ <div><label class="plabel">Age</label><input class="pinp" id="p-age" type="number" placeholder="e.g. 52"/></div>
291
+ <div><label class="plabel">Sex</label>
292
+ <select class="pinp" id="p-sex">
293
+ <option value="">Not specified</option>
294
+ <option>Female</option><option>Male</option><option>Other</option>
295
+ </select>
296
+ </div>
297
+ <div><label class="plabel">Previous scans</label><input class="pinp" id="p-scans" placeholder="e.g. Mammogram 2023 β€” normal"/></div>
298
+ </div>
299
+ <div style="margin-top:10px"><label class="plabel">Medical history</label><textarea class="ptextarea" id="p-history" placeholder="e.g. Family history of breast cancer, BRCA1 carrier, previous lumpectomy 2020..."></textarea></div>
300
+ <div style="margin-top:10px"><label class="plabel">Current symptoms / reason for scan</label><textarea class="ptextarea" id="p-symptoms" placeholder="e.g. Palpable lump in upper outer quadrant, noticed 3 weeks ago..."></textarea></div>
301
+ </div>
302
+ </div>
303
+
304
+ <div id="t-upload">
305
+ <div class="upload-zone" ondragover="event.preventDefault()" ondrop="event.preventDefault();loadFile(event.dataTransfer.files[0])">
306
+ <input type="file" accept="image/*" onchange="loadFile(this.files[0])"/>
307
+ <div style="font-size:32px;margin-bottom:10px">πŸ“€</div>
308
+ <div style="font-size:14px;font-weight:600;margin-bottom:6px">Drop a histopathology patch here</div>
309
+ <div style="font-size:12px;color:var(--muted)">PNG Β· JPG Β· TIFF Β· BMP</div>
310
+ </div>
311
+ <div id="preview-wrap" class="hidden mt16">
312
+ <img id="preview-img" class="simg" alt="Uploaded scan"/>
313
+ <div class="card-sm mt8" style="font-size:12px;color:var(--muted)" id="preview-name"></div>
314
+ </div>
315
+ </div>
316
+ <div id="t-overlay" class="hidden">
317
+ <div id="oph" class="sph"><div style="font-size:32px">🌑</div><div>Grad-CAM overlay</div><div style="font-size:11px;opacity:.6">Run analysis first</div></div>
318
+ <img id="oimg" class="simg hidden" alt="Overlay"/>
319
+ <div id="ostats" class="g2 mt12 hidden">
320
+ <div class="card-sm"><div style="font-size:10px;color:var(--muted);margin-bottom:3px">Mean activation</div><div class="mono" id="hmean" style="font-size:15px;font-weight:600">β€”</div></div>
321
+ <div class="card-sm"><div style="font-size:10px;color:var(--muted);margin-bottom:3px">Peak activation</div><div class="mono" id="hmax" style="font-size:15px;font-weight:600">β€”</div></div>
322
+ </div>
323
+ </div>
324
+ <div id="t-raw" class="hidden">
325
+ <div id="rph" class="sph"><div style="font-size:32px">πŸ“Š</div><div>Raw heatmap</div><div style="font-size:11px;opacity:.6">Greyscale activation map</div></div>
326
+ <img id="rimg" class="simg hidden" alt="Heatmap"/>
327
+ </div>
328
+ <div class="f g8 mt16">
329
+ <button class="btn-p" id="abtn" style="flex:1" onclick="runAnalysis()" disabled>Analyse scan</button>
330
+ <button class="btn-o" onclick="resetAll()">Reset</button>
331
+ </div>
332
+ <div id="errbox" class="ebox hidden"></div>
333
+ </div>
334
+
335
+ <div>
336
+ <div id="rph2" class="card f ac" style="min-height:420px;justify-content:center;flex-direction:column;gap:8px;color:var(--muted)">
337
+ <div style="font-size:32px">πŸ€–</div>
338
+ <div style="font-size:14px">Upload a scan and click Analyse</div>
339
+ <div style="font-size:12px;opacity:.6">Results appear here</div>
340
+ </div>
341
+ <div id="rpanel" class="hidden">
342
+ <div class="card">
343
+ <div class="slabel">AI prediction</div>
344
+ <div class="f ac jb mt8">
345
+ <div id="pbadge"></div>
346
+ <div class="mono" id="pconf" style="font-size:26px;font-weight:700"></div>
347
+ </div>
348
+ <div style="font-size:11px;color:var(--muted);margin:10px 0 5px">Confidence</div>
349
+ <div class="cbar"><div class="cfill" id="cfill" style="width:0%"></div></div>
350
+ <div class="g2 mt16" style="gap:10px">
351
+ <div><div style="font-size:11px;color:var(--muted);margin-bottom:3px">Logit Β· benign</div><div class="mono" id="lg0" style="font-size:14px;font-weight:600">β€”</div></div>
352
+ <div><div style="font-size:11px;color:var(--muted);margin-bottom:3px">Logit Β· malignant</div><div class="mono" id="lg1" style="font-size:14px;font-weight:600">β€”</div></div>
353
+ </div>
354
+ </div>
355
+ <div class="card mt16">
356
+ <div class="slabel">LLM explanation</div>
357
+ <div class="f ac g8 mt8 mb12" style="margin-bottom:12px">
358
+ <div class="dlive"></div>
359
+ <span class="mono" style="font-size:11px;color:var(--muted)" id="englab">template</span>
360
+ <span style="font-size:11px;color:var(--muted)" id="audlab"></span>
361
+ </div>
362
+ <div id="rsum" style="font-size:13px;line-height:1.65;padding-bottom:12px;border-bottom:.5px solid var(--border)"></div>
363
+ <div id="rdet" class="mt12" style="font-size:12px;color:var(--muted);line-height:1.6"></div>
364
+ <div id="rdisc" class="mt12" style="font-size:11px;color:#F09595;background:var(--red-dim);padding:8px 12px;border-radius:6px;border:.5px solid rgba(226,75,74,.2)"></div>
365
+ </div>
366
+ <div id="spatcard" class="card mt16 hidden">
367
+ <div class="slabel">Spatial analysis</div>
368
+ <div id="rspat" class="mt8" style="font-size:12px;color:var(--muted);line-height:1.6"></div>
369
+ </div>
370
+ <div class="mt12">
371
+ <button class="chip" onclick="document.getElementById('rawjson').classList.toggle('hidden')">Show raw JSON</button>
372
+ <pre id="rawjson" class="hidden mono" style="margin-top:10px;font-size:10px;color:var(--muted);background:var(--bg3);border:.5px solid var(--border);border-radius:7px;padding:12px;overflow-x:auto;white-space:pre-wrap;word-break:break-all;max-height:200px;overflow-y:auto"></pre>
373
+ </div>
374
+ </div>
375
+ </div>
376
+ </div>
377
+
378
+ <!-- Inline chat -->
379
+ <div id="chatwrap" class="hidden">
380
+ <div class="chat-panel">
381
+ <div class="chat-head">
382
+ <div class="f ac g10">
383
+ <div style="width:28px;height:28px;border-radius:7px;background:var(--teal-dim);display:flex;align-items:center;justify-content:center;font-size:13px">πŸ€–</div>
384
+ <div>
385
+ <div style="font-weight:600;font-size:13px">AI assistant β€” ask follow-up questions about this scan</div>
386
+ <div class="f ac g6 mt8" style="margin-top:4px"><div class="dlive"></div><span class="mono" style="font-size:10px;color:var(--muted)" id="chatmode">Clinician mode</span></div>
387
+ </div>
388
+ </div>
389
+ <div id="chipbar" class="f g6" style="flex-wrap:wrap;justify-content:flex-end;max-width:440px"></div>
390
+ </div>
391
+ <div class="chat-body" id="chatbody"></div>
392
+ <div class="chat-foot">
393
+ <input class="cinp" id="chatin" placeholder="Ask a follow-up question about this scan..." onkeydown="if(event.key==='Enter')sendChat()"/>
394
+ <button class="btn-p" style="padding:8px 16px;font-size:12px" onclick="sendChat()">Ask</button>
395
+ </div>
396
+ </div>
397
+ </div>
398
+ <div style="margin-top:24px">
399
+ <div class="slabel">Try a sample β€” click to analyse</div>
400
+ <div class="sample-row">
401
+ <div class="sample" onclick="loadSample('samples/histo-tumor.png','histo-tumor.png','histo')">
402
+ <img src="samples/histo-tumor.png" alt="tumor patch"/><span>Tumor patch</span>
403
+ </div>
404
+ <div class="sample" onclick="loadSample('samples/histo-normal.png','histo-normal.png','histo')">
405
+ <img src="samples/histo-normal.png" alt="normal patch"/><span>Normal patch</span>
406
+ </div>
407
+ </div>
408
+ </div>
409
+ </div>
410
+ <div id="dash-mammo" class="hidden">
411
+ <div style="display:grid;grid-template-columns:1fr 1fr;gap:24px;align-items:start">
412
+ <div>
413
+ <div class="tab-bar">
414
+ <button class="tab active" onclick="switchMammoTab('upload',this)">Upload mammogram</button>
415
+ <button class="tab" onclick="switchMammoTab('overlay',this)">Grad-CAM overlay</button>
416
+ <button class="tab" onclick="switchMammoTab('raw',this)">Raw heatmap</button>
417
+ </div>
418
+ <div id="mt-upload">
419
+ <div class="upload-zone" ondragover="event.preventDefault()" ondrop="event.preventDefault();loadMammo(event.dataTransfer.files[0])">
420
+ <input type="file" accept="image/*,.dcm,.dicom" onchange="loadMammo(this.files[0])"/>
421
+ <svg class="svgi" style="width:30px;height:30px;margin-bottom:10px;stroke:var(--teal-lt)" viewBox="0 0 24 24"><path d="M12 16V4m0 0L8 8m4-4 4 4"/><path d="M4 16v2a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2v-2"/></svg>
422
+ <div style="font-size:14px;font-weight:600;margin-bottom:6px">Drop a mammogram here</div>
423
+ <div style="font-size:12px;color:var(--muted)">PNG Β· JPG Β· TIFF Β· DICOM (.dcm)</div>
424
+ </div>
425
+ <div id="mammo-preview-wrap" class="hidden mt16">
426
+ <img id="mammo-preview-img" class="simg" alt="Mammogram"/>
427
+ <div class="card-sm mt8" style="font-size:12px;color:var(--muted)" id="mammo-preview-name"></div>
428
+ </div>
429
+ </div>
430
+ <div id="mt-overlay" class="hidden">
431
+ <div id="moph" class="sph"><div style="font-size:32px">🌑</div><div>Grad-CAM overlay</div><div style="font-size:11px;opacity:.6">Run analysis first</div></div>
432
+ <img id="moimg" class="simg hidden" alt="Mammogram overlay"/>
433
+ </div>
434
+ <div id="mt-raw" class="hidden">
435
+ <div id="mrph" class="sph"><div style="font-size:32px">πŸ“Š</div><div>Raw heatmap</div></div>
436
+ <img id="mrimg" class="simg hidden" alt="Mammogram heatmap"/>
437
+ </div>
438
+ <div class="f g8 mt16">
439
+ <button class="btn-p" id="mammo-btn" style="flex:1" onclick="runMammo()" disabled>Analyse mammogram</button>
440
+ <button class="btn-o" onclick="resetMammo()">Reset</button>
441
+ </div>
442
+ <div id="mammo-err" class="ebox hidden"></div>
443
+ </div>
444
+
445
+ <div>
446
+ <div id="mammo-rph" class="card f ac" style="min-height:420px;justify-content:center;flex-direction:column;gap:10px;color:var(--muted)">
447
+ <svg class="svgi" style="width:34px;height:34px;stroke:var(--muted)" viewBox="0 0 24 24"><rect x="3" y="3" width="18" height="18" rx="2"/><path d="M3 9h18M9 3v18"/></svg>
448
+ <div style="font-size:14px">Upload a mammogram and click Analyse</div>
449
+ <div style="font-size:12px;opacity:.6">Supports DICOM, PNG, and JPG</div>
450
+ </div>
451
+ <div id="mammo-rpanel" class="hidden">
452
+ <div class="card">
453
+ <div class="slabel">AI prediction</div>
454
+ <div id="mammo-banner" class="result-banner mt8">
455
+ <div class="rb-verdict" id="mammo-pbadge">β€”</div>
456
+ <div class="rb-conf"><div class="rb-num" id="mammo-pconf">β€”</div><div class="rb-lbl">confidence</div></div>
457
+ </div>
458
+ <div class="cbar mt16"><div class="cfill" id="mammo-cfill" style="width:0%"></div></div>
459
+ <div class="model-chip mt12" id="mammo-modelchip">AUC 0.84 Β· Sens 70% Β· Spec 82% Β· RSNA validation</div>
460
+ <div class="mt12 card-sm" style="background:var(--bg3)">
461
+ <div style="font-size:11px;color:var(--muted);margin-bottom:4px">BI-RADS suggestion</div>
462
+ <div id="mammo-birads" style="font-weight:600;font-size:13px"></div>
463
+ </div>
464
+ <details class="adv mt12">
465
+ <summary>Advanced β€” raw model outputs</summary>
466
+ <div class="g2" style="gap:10px;margin-top:4px">
467
+ <div><div style="font-size:11px;color:var(--muted);margin-bottom:3px">Logit Β· benign</div><div class="mono" id="mammo-lg0" style="font-size:14px;font-weight:600">β€”</div></div>
468
+ <div><div style="font-size:11px;color:var(--muted);margin-bottom:3px">Logit Β· malignant</div><div class="mono" id="mammo-lg1" style="font-size:14px;font-weight:600">β€”</div></div>
469
+ </div>
470
+ </details>
471
+ </div>
472
+ <div class="card mt16">
473
+ <div class="slabel">LLM explanation</div>
474
+ <div class="f ac g8 mt8" style="margin-bottom:12px">
475
+ <div class="dlive"></div>
476
+ <span class="mono" style="font-size:11px;color:var(--muted)" id="mammo-eng">AI summary</span>
477
+ </div>
478
+ <div id="mammo-sum" style="font-size:13px;line-height:1.65;padding-bottom:12px;border-bottom:.5px solid var(--border)"></div>
479
+ <div id="mammo-det" class="mt12" style="font-size:12px;color:var(--muted);line-height:1.6"></div>
480
+ <div id="mammo-disc" class="mt12" style="font-size:11px;color:#F09595;background:var(--red-dim);padding:8px 12px;border-radius:6px;border:.5px solid rgba(226,75,74,.2)"></div>
481
+ </div>
482
+ </div>
483
+ </div>
484
+ </div>
485
+
486
+ <div id="mammochatwrap" class="hidden" style="margin-top:24px">
487
+ <div class="chat-panel">
488
+ <div class="chat-head">
489
+ <div class="f ac g10">
490
+ <div style="width:28px;height:28px;border-radius:7px;background:var(--teal-dim);display:flex;align-items:center;justify-content:center"><svg class="svgi" viewBox="0 0 24 24"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg></div>
491
+ <div>
492
+ <div style="font-weight:600;font-size:13px">AI assistant β€” ask about this mammogram</div>
493
+ <div class="f ac g6" style="margin-top:4px"><div class="dlive"></div><span class="mono" style="font-size:10px;color:var(--muted)">powered by gpt-oss-120b</span></div>
494
+ </div>
495
+ </div>
496
+ <div class="f g6" style="flex-wrap:wrap;justify-content:flex-end;max-width:440px">
497
+ <button class="chip" onclick="askMammo('Why was this mammogram classified this way?')">Why this result?</button>
498
+ <button class="chip" onclick="askMammo('What does the suggested BI-RADS category mean for next steps?')">BI-RADS</button>
499
+ <button class="chip" onclick="askMammo('What is the Grad-CAM heatmap showing here?')">Heatmap</button>
500
+ <button class="chip" onclick="askMammo('Given 70% sensitivity, how should this result be interpreted cautiously?')">Reliability</button>
501
+ </div>
502
+ </div>
503
+ <div class="chat-body" id="mammochatbody"></div>
504
+ <div class="chat-foot">
505
+ <input class="cinp" id="mammochatin" placeholder="Ask a follow-up about this mammogram..." onkeydown="if(event.key==='Enter')sendMammoChat()"/>
506
+ <button class="btn-p" style="padding:8px 16px;font-size:12px" onclick="sendMammoChat()">Ask</button>
507
+ </div>
508
+ </div>
509
+ </div>
510
+
511
+ <!-- Info panel -->
512
+ <div style="display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin-top:32px">
513
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><rect x="3" y="3" width="18" height="18" rx="2"/><path d="M3 9h18M9 3v18"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">EfficientNet-B4</div><div style="font-size:12px;color:var(--muted);line-height:1.5">19M parameter CNN scaled for 512Γ—512 full-field mammography. RSNA 2022 competition-proven architecture.</div></div>
514
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><rect x="5" y="3" width="14" height="18" rx="2"/><path d="M9 3h6v3H9zM8 11h8M8 15h6"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">BI-RADS scoring</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Automatic BI-RADS category suggestion (2–5) based on malignancy probability. Requires radiologist confirmation.</div></div>
515
+ <div class="fcard"><div class="ficon"><svg class="svgi" viewBox="0 0 24 24"><path d="M3 7a2 2 0 0 1 2-2h4l2 2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2z"/></svg></div><div style="font-weight:600;font-size:14px;margin-bottom:6px">DICOM support</div><div style="font-size:12px;color:var(--muted);line-height:1.5">Native DICOM (.dcm) loading with VOI LUT windowing. Also accepts PNG, JPG, and TIFF formats.</div></div>
516
+ </div>
517
+
518
+ <div class="card mt20" style="background:var(--bg3)">
519
+ <div style="font-size:12px;color:var(--muted);line-height:1.6">
520
+ <strong style="color:var(--text)">Research &amp; educational use only.</strong>
521
+ This EfficientNet-B4 ensemble (0.84 AUC, ~70% sensitivity / ~82% specificity on RSNA validation) is a research prototype β€” not a medical device and not a diagnosis. All outputs must be reviewed by a qualified radiologist.
522
+ </div>
523
+ </div>
524
+
525
+ <div style="margin-top:24px">
526
+ <div class="slabel">Try a sample β€” click to analyse</div>
527
+ <div class="sample-row">
528
+ <div class="sample" onclick="loadSample('samples/mammo-cancer.png','mammo-cancer.png','mammo')">
529
+ <img src="samples/mammo-cancer.png" alt="cancer case"/><span>Cancer case</span>
530
+ </div>
531
+ <div class="sample" onclick="loadSample('samples/mammo-benign.png','mammo-benign.png','mammo')">
532
+ <img src="samples/mammo-benign.png" alt="benign case"/><span>Benign case</span>
533
+ </div>
534
+ </div>
535
+ </div>
536
+ </div>
537
+ </div>
538
+
539
+ <!-- ASSISTANT -->
540
+ <div id="v-assistant" class="page hidden">
541
+ <div style="display:grid;grid-template-columns:1fr 300px;gap:20px;align-items:start">
542
+ <div class="card" style="padding:0;overflow:hidden;display:flex;flex-direction:column;height:560px">
543
+ <div style="padding:14px 18px;border-bottom:.5px solid var(--border);display:flex;align-items:center;gap:10px">
544
+ <div style="width:32px;height:32px;border-radius:8px;background:var(--teal-dim);display:flex;align-items:center;justify-content:center;font-size:16px">πŸ€–</div>
545
+ <div>
546
+ <div style="font-weight:600;font-size:14px">AI radiology assistant</div>
547
+ <div class="f ac g6 mt8" style="margin-top:4px"><div class="dlive"></div><span class="mono" style="font-size:10px;color:var(--muted)">DenseNet-121 Β· FLAN-T5 Β· Grad-CAM</span></div>
548
+ </div>
549
+ </div>
550
+ <div class="chat-body" id="asstbody" style="height:auto;flex:1"></div>
551
+ <div class="chip-row">
552
+ <button class="chip" onclick="askAsst('Why might a scan be classified as malignant?')">Why malignant?</button>
553
+ <button class="chip" onclick="askAsst('Explain what Grad-CAM shows')">Grad-CAM</button>
554
+ <button class="chip" onclick="askAsst('What does 87.5% sensitivity mean?')">Sensitivity</button>
555
+ <button class="chip" onclick="askAsst('How does FLAN-T5 generate explanations?')">How LLM works</button>
556
+ </div>
557
+ <div class="chat-foot">
558
+ <input class="cinp" id="asstinp" placeholder="Ask about the model, training, or a scan..." onkeydown="if(event.key==='Enter')sendAsst()"/>
559
+ <button class="btn-p" style="padding:8px 16px;font-size:12px" onclick="sendAsst()">Send</button>
560
+ </div>
561
+ </div>
562
+ <div class="fc g12">
563
+ <div class="card">
564
+ <div class="slabel">Quick topics</div>
565
+ <div class="fc g4 mt8">
566
+ <div onclick="askAsst('Explain the DenseNet-121 architecture')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί DenseNet-121 architecture</div>
567
+ <div onclick="askAsst('What is StainJitter and why does it help?')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί StainJitter augmentation</div>
568
+ <div onclick="askAsst('Explain OneCycleLR and why it helped training')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί OneCycleLR scheduler</div>
569
+ <div onclick="askAsst('Explain PCam dataset deduplication')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί PCam deduplication</div>
570
+ <div onclick="askAsst('How does Mixup augmentation work?')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί Mixup augmentation</div>
571
+ <div onclick="askAsst('What improved sensitivity from 80% to 87.5%?')" style="padding:7px 8px;border-radius:6px;cursor:pointer;font-size:12px;color:var(--muted)" onmouseover="this.style.background='var(--bg3)'" onmouseout="this.style.background='transparent'">β€Ί Training improvements</div>
572
+ </div>
573
+ </div>
574
+ <div class="card">
575
+ <div class="slabel">Connect live scan</div>
576
+ <p style="font-size:12px;color:var(--muted);margin-top:8px;line-height:1.6">Run an analysis in the dashboard first, then return here to ask questions about that specific scan.</p>
577
+ <button class="btn-p" style="width:100%;margin-top:12px;font-size:12px" onclick="show('dashboard')">Open dashboard β†’</button>
578
+ </div>
579
+ </div>
580
+ </div>
581
+ </div>
582
+
583
+ <!-- RESEARCH -->
584
+ <div id="v-research" class="page hidden">
585
+ <div class="slabel">Model card</div>
586
+ <h2 class="h2" style="margin-bottom:28px">Architecture, datasets &amp; methodology</h2>
587
+
588
+ <div class="slabel" style="margin-bottom:12px;color:var(--teal-lt)">Histopathology module</div>
589
+ <div class="g2">
590
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">DenseNet-121 backbone</div><div style="font-size:13px;color:var(--muted);line-height:1.6">~7.2M-parameter CNN pretrained on ImageNet. Dense-block feature reuse across layers. Custom head: BN→Dropout(0.4)→Linear(1024→256)→ReLU→BN→Dropout(0.3)→Linear(256→2).</div></div>
591
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">PatchCamelyon (PCam)</div><div style="font-size:13px;color:var(--muted);line-height:1.6">H&amp;E patches from Camelyon16 slides. After MD5 deduplication: 220,025 unique samples (benign 130,908 / malignant 89,117 β†’ pos_weight 1.469). Patient-disjoint train / val / test splits.</div></div>
592
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">Training methodology</div><div style="font-size:13px;color:var(--muted);line-height:1.6">OneCycleLR (max_lr 3e-3, pct_start 0.3) Β· Mixup (Ξ± 0.4) Β· StainJitter (HED, strength 0.05) Β· label smoothing 0.1 Β· checkpoint by validation sensitivity.</div></div>
593
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">Histopathology results</div><div style="font-size:13px;color:var(--muted);line-height:1.6">Held-out test: 88.0% accuracy, 87.5% sensitivity (val sensitivity 90.3%, best epoch 13/20). Improvement vs baseline +6.8pp (80.7% β†’ 87.5%).</div></div>
594
+ </div>
595
+
596
+ <div class="slabel" style="margin:28px 0 12px;color:var(--teal-lt)">Mammography module</div>
597
+ <div class="g2">
598
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">EfficientNet-B4 ensemble</div><div style="font-size:13px;color:var(--muted);line-height:1.6">Three EfficientNet-B4 models (seeds 42 / 123 / 999), probabilities averaged at inference. Member AUCs 0.7989 / 0.8254 / 0.8083.</div></div>
599
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">RSNA dataset &amp; preprocessing</div><div style="font-size:13px;color:var(--muted);line-height:1.6">RSNA Screening Mammography (2022). DICOM decoding with VOI-LUT windowing, resized for full-field input. Held-out patient-level validation split.</div></div>
600
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">Mammography results</div><div style="font-size:13px;color:var(--muted);line-height:1.6">Ensemble patient-level AUC 0.8443 Β· sensitivity 70.1% Β· specificity 82.4% (threshold 0.50). Suggested BI-RADS 2–5 derived from malignancy probability.</div></div>
601
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">Negative ablations</div><div style="font-size:13px;color:var(--muted);line-height:1.6">Breast-ROI cropping reduced AUC (~0.14 drop, aspect-ratio distortion); adding external CBIS-DDSM data hurt (film-vs-digital domain shift). The un-cropped, RSNA-only ensemble is final.</div></div>
602
+ </div>
603
+
604
+ <div class="slabel" style="margin:28px 0 12px;color:var(--teal-lt)">Shared layers β€” both modules</div>
605
+ <div class="g2">
606
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">Grad-CAM explainability</div><div style="font-size:13px;color:var(--muted);line-height:1.6">Gradients backpropagated w.r.t. the predicted class at the final conv block β†’ global-average-pooled channel weights β†’ weighted feature sum β†’ ReLU β†’ bilinear upsample to input resolution.</div></div>
607
+ <div class="fcard"><div style="font-weight:600;font-size:15px;margin-bottom:10px">LLM explanation layer</div><div style="font-size:13px;color:var(--muted);line-height:1.6">FLAN-T5-large / BioMedLM / Llama 3.2 with a deterministic template engine. Three audience modes (clinician, researcher, patient) with genuinely different content.</div></div>
608
+ </div>
609
+ </div>
610
+
611
+ <footer class="site-footer">
612
+ <div class="footer-disc">Research use only. Not a medical device and not a substitute for professional diagnosis.</div>
613
+ <div class="footer-row">
614
+ <div>
615
+ <div style="font-weight:600;color:var(--text)">MedAI β€” dual-modality breast cancer detection</div>
616
+ <div style="font-size:12px;color:var(--muted);margin-top:3px">Histopathology (DenseNet-121) + mammography (EfficientNet-B4 ensemble) Β· by Lateef Olatunji</div>
617
+ </div>
618
+ <div class="f g16 ac" style="flex-wrap:wrap;font-size:13px">
619
+ <a href="https://github.com/Relixsx" target="_blank" rel="noopener">GitHub</a>
620
+ <a href="mailto:relixsx@gmail.com">Contact</a>
621
+ <a href="#" onclick="show('research');return false;">Model card</a>
622
+ <span style="color:var(--muted)">MIT License</span>
623
+ </div>
624
+ </div>
625
+ </footer>
626
+
627
+ <script>
628
+ // ── Backend API base URL ───────────────────────────────────────────
629
+ // Local dev: leave blank β€” the app uses http://localhost:8000.
630
+ // Deployment: set to your backend's public URL (e.g. "https://api.example.com"),
631
+ // or leave blank to call the same origin the page is served from.
632
+ window.MEDAI_API_URL =
633
+ (location.hostname === "localhost" || location.hostname === "127.0.0.1" || location.protocol === "file:")
634
+ ? "" // local dev β†’ uses http://localhost:8000
635
+ : "https://relixsx-medai.hf.space"; // deployed β†’ Hugging Face Space backend
636
+ </script>
637
+
638
+ <script>
639
+ var currentFile = null;
640
+ var scanResult = null;
641
+
642
+ function togglePatient(){
643
+ var body=document.getElementById('pform-body');
644
+ var icon=document.getElementById('ptoggle-icon');
645
+ var hidden=body.classList.contains('hidden');
646
+ body.classList.toggle('hidden',!hidden);
647
+ icon.textContent=hidden?'β–Ό':'β–Ά';
648
+ }
649
+
650
+ var mammoFile = null;
651
+ var mammoResult = null;
652
+
653
+ function switchMammoTab(name,btn){
654
+ ['upload','overlay','raw'].forEach(function(t){document.getElementById('mt-'+t).classList.toggle('hidden',t!==name);});
655
+ document.querySelectorAll('#dash-mammo .tab').forEach(function(b){b.classList.remove('active');});
656
+ btn.classList.add('active');
657
+ }
658
+
659
+ function loadMammo(file){
660
+ if(!file)return;
661
+ mammoFile=file;
662
+ var r=new FileReader();
663
+ r.onload=function(e){
664
+ document.getElementById('mammo-preview-img').src=e.target.result;
665
+ document.getElementById('mammo-preview-name').textContent=file.name+' Β· '+(file.size/1024).toFixed(1)+' KB';
666
+ document.getElementById('mammo-preview-wrap').classList.remove('hidden');
667
+ document.getElementById('mammo-btn').disabled=false;
668
+ };
669
+ r.readAsDataURL(file);
670
+ document.getElementById('mammo-rph').style.display='';
671
+ document.getElementById('mammo-rpanel').classList.add('hidden');
672
+ }
673
+
674
+ function resetMammo(){
675
+ mammoFile=null; mammoResult=null;
676
+ document.getElementById('mammo-preview-wrap').classList.add('hidden');
677
+ document.getElementById('mammo-btn').disabled=true;
678
+ document.getElementById('mammo-rph').style.display='';
679
+ document.getElementById('mammo-rpanel').classList.add('hidden');
680
+ document.getElementById('mammo-err').classList.add('hidden');
681
+ ['mo','mr'].forEach(function(p){
682
+ document.getElementById(p+'img').classList.add('hidden');
683
+ document.getElementById(p+'ph').style.display='';
684
+ });
685
+ }
686
+
687
+ function runMammo(){
688
+ if(!mammoFile)return;
689
+ var btn=document.getElementById('mammo-btn'), err=document.getElementById('mammo-err');
690
+ btn.disabled=true; btn.textContent='Analysing...'; err.classList.add('hidden');
691
+ var aud=document.getElementById('audience').value;
692
+ var fd=new FormData(); fd.append('file',mammoFile);
693
+ fetch(getApi()+'/mammogram/visual?audience='+aud,{method:'POST',body:fd})
694
+ .then(function(r){
695
+ if(!r.ok)return r.json().then(function(e){throw new Error(e.detail||r.statusText);});
696
+ return r.json();
697
+ })
698
+ .then(function(d){mammoResult=d; renderMammo(d);})
699
+ .catch(function(e){err.textContent='Error: '+e.message; err.classList.remove('hidden');})
700
+ .finally(function(){btn.disabled=false; btn.textContent='Analyse mammogram';});
701
+ }
702
+
703
+ function renderMammo(d){
704
+ var mal=d.prediction==='malignant';
705
+ var banner=document.getElementById('mammo-banner');
706
+ banner.className='result-banner mt8 '+(mal?'suspicious':'benign');
707
+ var icon=mal
708
+ ? '<svg class="svgi" style="stroke:#F4A6A6" viewBox="0 0 24 24"><path d="M12 9v4m0 4h.01M10.3 3.9 1.8 18a2 2 0 0 0 1.7 3h17a2 2 0 0 0 1.7-3L13.7 3.9a2 2 0 0 0-3.4 0z"/></svg>'
709
+ : '<svg class="svgi" style="stroke:var(--teal-lt)" viewBox="0 0 24 24"><path d="M20 6 9 17l-5-5"/></svg>';
710
+ document.getElementById('mammo-pbadge').innerHTML=icon+(mal?'Suspicious':'Benign');
711
+ var pct=(d.confidence*100).toFixed(1)+'%';
712
+ document.getElementById('mammo-pconf').textContent=pct;
713
+ var cf=document.getElementById('mammo-cfill'); cf.style.width=pct;
714
+ cf.style.background=mal?'var(--red)':'var(--teal)';
715
+ document.getElementById('mammo-birads').textContent=d.birads||'β€”';
716
+ document.getElementById('mammo-lg0').textContent=d.logits[0].toFixed(4);
717
+ document.getElementById('mammo-lg1').textContent=d.logits[1].toFixed(4);
718
+ document.getElementById('mammo-eng').textContent=(d.engine && d.engine!=='template')?d.engine:'AI summary';
719
+ document.getElementById('mammo-sum').textContent=d.summary||'';
720
+ document.getElementById('mammo-det').textContent=d.detail||'';
721
+ document.getElementById('mammo-disc').textContent=d.disclaimer||'';
722
+ var mcw=document.getElementById('mammochatwrap');
723
+ if(mcw){ mcw.classList.remove('hidden'); mammoChatHistory=[]; document.getElementById('mammochatbody').innerHTML=''; }
724
+ if(d.overlay_b64){
725
+ var oi=document.getElementById('moimg'); oi.src='data:image/png;base64,'+d.overlay_b64; oi.classList.remove('hidden');
726
+ document.getElementById('moph').style.display='none';
727
+ }
728
+ if(d.heatmap_b64){
729
+ var ri=document.getElementById('mrimg'); ri.src='data:image/png;base64,'+d.heatmap_b64; ri.classList.remove('hidden');
730
+ document.getElementById('mrph').style.display='none';
731
+ }
732
+ document.getElementById('mammo-rph').style.display='none';
733
+ document.getElementById('mammo-rpanel').classList.remove('hidden');
734
+ }
735
+
736
+ function getApi(){
737
+ var u=(window.MEDAI_API_URL||'').replace(/\/$/,'');
738
+ if(u) return u;
739
+ var h=location.hostname;
740
+ if(location.protocol==='file:' || h==='localhost' || h==='127.0.0.1') return 'http://localhost:8000';
741
+ return location.origin;
742
+ }
743
+ window.addEventListener('load', checkHealth);
744
+
745
+ function show(name){
746
+ ['platform','dashboard','assistant','research'].forEach(function(v){
747
+ document.getElementById('v-'+v).classList.toggle('hidden',v!==name);
748
+ if(document.getElementById('nb-'+v))document.getElementById('nb-'+v).classList.toggle('active',v===name);
749
+ });
750
+ }
751
+
752
+ function switchModality(name){
753
+ document.getElementById('dash-histo').classList.toggle('hidden',name!=='histo');
754
+ document.getElementById('dash-mammo').classList.toggle('hidden',name!=='mammo');
755
+ document.getElementById('mod-histo').classList.toggle('active',name==='histo');
756
+ document.getElementById('mod-mammo').classList.toggle('active',name==='mammo');
757
+ var sub=document.getElementById('dash-sub');
758
+ if(sub)sub.textContent = name==='mammo'
759
+ ? 'EfficientNet-B4 mammogram classifier \u00b7 BI-RADS scoring \u00b7 Grad-CAM'
760
+ : 'Upload a histopathology patch to run the full pipeline';
761
+ }
762
+
763
+ function checkHealth(){
764
+ var dot=document.getElementById('apidot'), st=document.getElementById('apistatus');
765
+ if(dot){dot.style.background='#888'; dot.style.animation='none';}
766
+ if(st)st.textContent='Checking…';
767
+ fetch(getApi()+'/health')
768
+ .then(function(r){if(!r.ok)throw 0; return r.json();})
769
+ .then(function(){
770
+ if(dot){dot.style.background='#2E8BF5'; dot.style.animation='pulse 1.5s ease infinite';}
771
+ if(st)st.textContent='API online';
772
+ })
773
+ .catch(function(){
774
+ if(dot){dot.style.background='#E24B4A'; dot.style.animation='none';}
775
+ if(st)st.textContent='API offline';
776
+ });
777
+ }
778
+
779
+ function loadSample(url,name,modality){
780
+ fetch(url).then(function(r){
781
+ if(!r.ok)throw new Error('not found');
782
+ return r.blob();
783
+ }).then(function(blob){
784
+ var file=new File([blob],name,{type:blob.type||'image/png'});
785
+ if(modality==='mammo'){ if(typeof switchModality==='function')switchModality('mammo'); loadMammo(file); setTimeout(runMammo,200); }
786
+ else { if(typeof switchModality==='function')switchModality('histo'); loadFile(file); setTimeout(runAnalysis,200); }
787
+ window.scrollTo({top:0,behavior:'smooth'});
788
+ }).catch(function(){
789
+ alert('Could not load the sample image.\n\nIf you opened the page directly (file://), browsers block loading local images. Serve it over HTTP instead:\n\n cd frontend\n python3 -m http.server 5500\n\nthen open http://localhost:5500\n\nAlso make sure frontend/samples/ contains the image files.');
790
+ });
791
+ }
792
+
793
+ function loadFile(file){
794
+ if(!file)return;
795
+ currentFile=file;
796
+ var r=new FileReader();
797
+ r.onload=function(e){
798
+ document.getElementById('preview-img').src=e.target.result;
799
+ document.getElementById('preview-name').textContent=file.name+' Β· '+(file.size/1024).toFixed(1)+' KB';
800
+ document.getElementById('preview-wrap').classList.remove('hidden');
801
+ document.getElementById('abtn').disabled=false;
802
+ };
803
+ r.readAsDataURL(file);
804
+ resetResults();
805
+ }
806
+
807
+ function switchTab(name,btn){
808
+ ['upload','overlay','raw'].forEach(function(t){document.getElementById('t-'+t).classList.toggle('hidden',t!==name);});
809
+ document.querySelectorAll('#dash-histo .tab').forEach(function(b){b.classList.remove('active');});
810
+ btn.classList.add('active');
811
+ }
812
+
813
+ function resetAll(){
814
+ currentFile=null; scanResult=null;
815
+ document.getElementById('preview-wrap').classList.add('hidden');
816
+ document.getElementById('abtn').disabled=true;
817
+ document.getElementById('chatwrap').classList.add('hidden');
818
+ resetResults();
819
+ ['o','r'].forEach(function(p){
820
+ document.getElementById(p+'img').classList.add('hidden');
821
+ document.getElementById(p+'ph').style.display='';
822
+ });
823
+ document.getElementById('ostats').classList.add('hidden');
824
+ }
825
+
826
+ function resetResults(){
827
+ document.getElementById('rph2').style.display='';
828
+ document.getElementById('rpanel').classList.add('hidden');
829
+ document.getElementById('errbox').classList.add('hidden');
830
+ document.getElementById('rawjson').classList.add('hidden');
831
+ }
832
+
833
+ function runAnalysis(){
834
+ if(!currentFile)return;
835
+ var btn=document.getElementById('abtn'), err=document.getElementById('errbox');
836
+ btn.disabled=true; btn.textContent='Analysing...'; err.classList.add('hidden');
837
+ var aud=document.getElementById('audience').value;
838
+ var fd=new FormData(); fd.append('file',currentFile);
839
+ fetch(getApi()+'/explain/visual?audience='+aud,{method:'POST',body:fd})
840
+ .then(function(r){
841
+ if(!r.ok)return r.json().then(function(e){throw new Error(e.detail||r.statusText);});
842
+ return r.json();
843
+ })
844
+ .then(function(d){scanResult=d; renderResults(d,aud);})
845
+ .catch(function(e){err.textContent='Error: '+e.message; err.classList.remove('hidden');})
846
+ .finally(function(){btn.disabled=false; btn.textContent='Analyse scan';});
847
+ }
848
+
849
+ function renderResults(d,aud){
850
+ var badge=document.getElementById('pbadge');
851
+ badge.innerHTML=d.prediction==='malignant'?'<span class="bdanger">⚠ Malignant</span>':'<span class="bsafe">βœ“ Benign</span>';
852
+ var pct=(d.confidence*100).toFixed(1)+'%';
853
+ var pc=document.getElementById('pconf'); pc.textContent=pct; pc.style.color=d.prediction==='malignant'?'var(--red)':'var(--teal)';
854
+ var cf=document.getElementById('cfill'); cf.style.width=pct; cf.style.background=d.prediction==='malignant'?'var(--red)':'var(--teal)';
855
+ document.getElementById('lg0').textContent=d.logits[0].toFixed(4);
856
+ document.getElementById('lg1').textContent=d.logits[1].toFixed(4);
857
+ document.getElementById('englab').textContent=d.engine;
858
+ document.getElementById('audlab').textContent='Β· '+aud;
859
+ document.getElementById('rsum').textContent=d.summary;
860
+ document.getElementById('rdet').textContent=d.detail;
861
+ document.getElementById('rdisc').textContent=d.disclaimer;
862
+ if(d.spatial_summary){document.getElementById('rspat').textContent=d.spatial_summary; document.getElementById('spatcard').classList.remove('hidden');}
863
+ if(d.overlay_b64){
864
+ var oi=document.getElementById('oimg'); oi.src='data:image/png;base64,'+d.overlay_b64; oi.classList.remove('hidden');
865
+ document.getElementById('oph').style.display='none'; document.getElementById('ostats').classList.remove('hidden');
866
+ document.getElementById('hmean').textContent=(d.heatmap_mean*100).toFixed(1)+'%';
867
+ document.getElementById('hmax').textContent=(d.heatmap_max*100).toFixed(1)+'%';
868
+ }
869
+ if(d.heatmap_b64){
870
+ var ri=document.getElementById('rimg'); ri.src='data:image/png;base64,'+d.heatmap_b64; ri.classList.remove('hidden');
871
+ document.getElementById('rph').style.display='none';
872
+ }
873
+ document.getElementById('rawjson').textContent=JSON.stringify(d,null,2);
874
+ document.getElementById('rph2').style.display='none';
875
+ document.getElementById('rpanel').classList.remove('hidden');
876
+ openChat(d,aud);
877
+ }
878
+
879
+ var CHIPS={
880
+ clinician:['Why this result?','BI-RADS score?','Biopsy needed?','Grad-CAM regions','Model accuracy'],
881
+ researcher:['Logit math','Grad-CAM method','Training details','Dataset stats','Calibration?'],
882
+ patient:['Should I worry?','What does this mean?','What happens next?','How accurate?','Explain heatmap'],
883
+ };
884
+
885
+ function openChat(result,aud){
886
+ document.getElementById('chatmode').textContent=aud.charAt(0).toUpperCase()+aud.slice(1)+' mode';
887
+ var bar=document.getElementById('chipbar'); bar.innerHTML='';
888
+ (CHIPS[aud]||CHIPS.clinician).forEach(function(q){
889
+ var b=document.createElement('button'); b.className='chip'; b.textContent=q;
890
+ b.onclick=function(){sendChatMsg(q);}; bar.appendChild(b);
891
+ });
892
+ var msgs=document.getElementById('chatbody'); msgs.innerHTML='';
893
+ var greets={
894
+ clinician:'Analysis complete. '+result.prediction.toUpperCase()+' at '+(result.confidence*100).toFixed(1)+'% confidence. What would you like to explore further?',
895
+ researcher:'Output: '+result.prediction.toUpperCase()+' | logits=['+result.logits.map(function(l){return l.toFixed(4);}).join(', ')+'] | P='+result.confidence.toFixed(4)+'. Ask me anything about the model internals.',
896
+ patient:result.prediction==='malignant'?"The AI flagged something in this sample. I'm here to help you understand what this means. What would you like to know?"
897
+ :"Good news β€” the AI found no signs of abnormal tissue. Do you have any questions?",
898
+ };
899
+ addCMsg(greets[aud]||greets.clinician,'ai');
900
+ document.getElementById('chatwrap').classList.remove('hidden');
901
+ document.getElementById('chatwrap').scrollIntoView({behavior:'smooth',block:'nearest'});
902
+ }
903
+
904
+ function addCMsg(text,role){
905
+ var msgs=document.getElementById('chatbody');
906
+ var d=document.createElement('div'); d.className=role==='ai'?'mai':'muser'; d.textContent=text;
907
+ msgs.appendChild(d); msgs.scrollTop=msgs.scrollHeight;
908
+ }
909
+
910
+ function showTypingC(){
911
+ var msgs=document.getElementById('chatbody');
912
+ var d=document.createElement('div'); d.className='mai'; d.id='typingc';
913
+ d.innerHTML='<span class="td"></span><span class="td"></span><span class="td"></span>';
914
+ msgs.appendChild(d); msgs.scrollTop=msgs.scrollHeight;
915
+ }
916
+ function removeTypingC(){var e=document.getElementById('typingc');if(e)e.remove();}
917
+
918
+ function sendChat(){
919
+ var inp=document.getElementById('chatin'), msg=inp.value.trim();
920
+ if(!msg||!scanResult)return; inp.value=''; sendChatMsg(msg);
921
+ }
922
+
923
+ function sendChatMsg(message){
924
+ if(!scanResult)return;
925
+ addCMsg(message,'user'); showTypingC();
926
+ var aud=document.getElementById('audience').value;
927
+ var patient={
928
+ name:document.getElementById('p-name').value,
929
+ age:parseInt(document.getElementById('p-age').value)||0,
930
+ sex:document.getElementById('p-sex').value,
931
+ medical_history:document.getElementById('p-history').value,
932
+ symptoms:document.getElementById('p-symptoms').value,
933
+ previous_scans:document.getElementById('p-scans').value,
934
+ };
935
+ fetch(getApi()+'/chat',{
936
+ method:'POST', headers:{'Content-Type':'application/json'},
937
+ body:JSON.stringify({message:message,audience:aud,prediction:scanResult.prediction,
938
+ confidence:scanResult.confidence,logits:scanResult.logits,
939
+ spatial_summary:scanResult.spatial_summary||'',history:[],patient:patient}),
940
+ })
941
+ .then(function(r){return r.json();})
942
+ .then(function(d){removeTypingC(); addCMsg(d.response||'No response.','ai');})
943
+ .catch(function(e){removeTypingC(); addCMsg('Error: '+e.message,'ai');});
944
+ }
945
+
946
+ var KB={
947
+ 'why might a scan be classified as malignant':'DenseNet-121 flags a scan as malignant when it detects feature patterns associated with cancerous tissue β€” irregular nuclear morphology, high cell density, distorted architecture. The Grad-CAM overlay shows exactly which spatial regions triggered these activations.',
948
+ 'explain what grad-cam shows':"Grad-CAM backpropagates the class score's gradient through the final convolutional layer (norm5, 1024Γ—7Γ—7 maps). Channel importance weights are computed via global average pooling of gradients, then used to create a weighted sum of feature maps. ReLU removes negative activations. The result is upsampled to 224Γ—224 and overlaid using a jet colourmap β€” red = high attention, blue = low attention.",
949
+ 'what does 87.5% sensitivity mean':'Sensitivity of 87.5% means the model correctly identifies 87.5% of actual malignant patches. The 12.5% are missed tumour patches (false negatives). For cancer screening, sensitivity is the most critical metric β€” missing a cancer is far worse than a false alarm.',
950
+ 'how does flan-t5 generate explanations':'FLAN-T5-large (~780MB) runs entirely locally. The LLMExplainer builds a structured prompt with prediction, confidence, logits, and Grad-CAM spatial findings. A deterministic TemplateEngine provides audience-specific structure; FLAN-T5 supplements with natural language.',
951
+ 'explain the densenet-121 architecture':'DenseNet-121 has 4 dense blocks where every layer receives feature maps from ALL preceding layers via concatenation. This enables feature reuse, reduces vanishing gradients, and allows learning both low-level and high-level features simultaneously.',
952
+ 'what is stainji':'StainJitter perturbs H&E stain concentrations in HED colour space using the Ruifrok & Johnston deconvolution matrix. It randomly scales and shifts each stain channel then reconstructs RGB, simulating real-world staining variation between labs.',
953
+ 'explain onecyclelr':'OneCycleLR ramps the learning rate from low up to max_lr (3e-3) over the first 30% of training, then cosine-anneals back to near zero β€” stepping every batch. This is why the model kept improving through epoch 13 instead of peaking at epoch 1-2.',
954
+ 'explain pcam dataset deduplication':'The original PCam dataset has 262,144 training images appearing perfectly 50/50 balanced. After MD5 dedup: 220,025 unique samples. Hidden truth: 42,119 duplicate malignant patches were removed, revealing the real distribution is 60% benign / 40% malignant (pos_weight=1.469).',
955
+ 'how does mixup augmentation work':'Mixup blends two training images: mixed = λ·imageA + (1-λ)·imageB with labels also blended. λ is sampled from Beta(0.4, 0.4). This forces the model to learn smooth decision boundaries rather than memorising hard boundaries around specific training examples.',
956
+ 'what improved sensitivity':'Three changes drove sensitivity from 80.7% to 87.5%: (1) PCam deduplication β€” fixed hidden class imbalance; (2) OneCycleLR β€” model found better minima and kept improving through epoch 13; (3) Checkpoint by val_sensitivity β€” stopped selecting low-sensitivity checkpoints.',
957
+ };
958
+
959
+ function askAsst(q){document.getElementById('asstinp').value=q; sendAsst();}
960
+
961
+ var asstHistory=[];
962
+
963
+ function fmtMd(t){
964
+ var s=t.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
965
+ var blocks=[];
966
+ s=s.replace(/```([\s\S]*?)```/g,function(m,c){
967
+ blocks.push('<pre><code>'+c.replace(/^\n+|\n+$/g,'')+'</code></pre>');
968
+ return '\u0001'+(blocks.length-1)+'\u0001';
969
+ });
970
+ function inline(x){
971
+ return x.replace(/\*\*([^*]+)\*\*/g,'<strong>$1</strong>')
972
+ .replace(/`([^`]+)`/g,'<code>$1</code>');
973
+ }
974
+ var lines=s.split('\n'), out=[], i=0;
975
+ while(i<lines.length){
976
+ var ln=lines[i], ph=ln.match(/^\u0001(\d+)\u0001$/);
977
+ if(ph){ out.push(blocks[+ph[1]]); i++; continue; }
978
+ var h=ln.match(/^(#{1,5})\s+(.*)$/);
979
+ if(h){ var lv=Math.min(h[1].length+2,5); out.push('<h'+lv+'>'+inline(h[2])+'</h'+lv+'>'); i++; continue; }
980
+ if(/^\s*[-*]\s+/.test(ln)){
981
+ var ul=[];
982
+ while(i<lines.length && /^\s*[-*]\s+/.test(lines[i])){ ul.push('<li>'+inline(lines[i].replace(/^\s*[-*]\s+/,''))+'</li>'); i++; }
983
+ out.push('<ul>'+ul.join('')+'</ul>'); continue;
984
+ }
985
+ if(/^\s*\d+\.\s+/.test(ln)){
986
+ var ol=[];
987
+ while(i<lines.length && /^\s*\d+\.\s+/.test(lines[i])){ ol.push('<li>'+inline(lines[i].replace(/^\s*\d+\.\s+/,''))+'</li>'); i++; }
988
+ out.push('<ol>'+ol.join('')+'</ol>'); continue;
989
+ }
990
+ if(ln.trim()===''){ i++; continue; }
991
+ var para=[ln]; i++;
992
+ while(i<lines.length && lines[i].trim()!=='' &&
993
+ !/^(#{1,5}\s|\s*[-*]\s|\s*\d+\.\s)/.test(lines[i]) &&
994
+ !/^\u0001\d+\u0001$/.test(lines[i])){ para.push(lines[i]); i++; }
995
+ out.push('<p>'+inline(para.join(' '))+'</p>');
996
+ }
997
+ return out.join('');
998
+ }
999
+
1000
+ function sendAsst(){
1001
+ var inp=document.getElementById('asstinp'), msg=inp.value.trim();
1002
+ if(!msg)return; inp.value='';
1003
+ addAMsg(msg,'user');
1004
+ asstHistory.push({role:'user',content:msg});
1005
+ var bubble=addAMsg('','ai'); bubble.classList.add('md'); bubble.textContent='…';
1006
+ fetch(getApi()+'/chat/stream',{
1007
+ method:'POST', headers:{'Content-Type':'application/json'},
1008
+ body:JSON.stringify({messages:asstHistory, audience:'clinician'})
1009
+ }).then(function(r){
1010
+ if(!r.ok||!r.body){ bubble.textContent='Error: '+(r.statusText||'no response'); return; }
1011
+ var reader=r.body.getReader(), dec=new TextDecoder(), acc='';
1012
+ bubble.textContent='';
1013
+ function pump(){
1014
+ return reader.read().then(function(res){
1015
+ if(res.done){ asstHistory.push({role:'assistant',content:acc}); return; }
1016
+ acc+=dec.decode(res.value,{stream:true});
1017
+ bubble.innerHTML=fmtMd(acc);
1018
+ var b=document.getElementById('asstbody'); b.scrollTop=b.scrollHeight;
1019
+ return pump();
1020
+ });
1021
+ }
1022
+ return pump();
1023
+ }).catch(function(e){ bubble.textContent='Error: '+e.message; });
1024
+ }
1025
+
1026
+ function addAMsg(text,role){
1027
+ var msgs=document.getElementById('asstbody');
1028
+ var d=document.createElement('div'); d.className=role==='ai'?'mai':'muser'; d.textContent=text;
1029
+ msgs.appendChild(d); msgs.scrollTop=msgs.scrollHeight;
1030
+ return d;
1031
+ }
1032
+
1033
+ var mammoChatHistory=[];
1034
+ function askMammo(q){ document.getElementById('mammochatin').value=q; sendMammoChat(); }
1035
+ function addMMsg(text,role){
1036
+ var b=document.getElementById('mammochatbody');
1037
+ var d=document.createElement('div'); d.className=role==='ai'?'mai':'muser'; d.textContent=text;
1038
+ b.appendChild(d); b.scrollTop=b.scrollHeight; return d;
1039
+ }
1040
+ function sendMammoChat(){
1041
+ var inp=document.getElementById('mammochatin'), msg=inp.value.trim();
1042
+ if(!msg||!mammoResult)return; inp.value='';
1043
+ addMMsg(msg,'user');
1044
+ mammoChatHistory.push({role:'user',content:msg});
1045
+ var bubble=addMMsg('','ai'); bubble.classList.add('md'); bubble.textContent='…';
1046
+ var aud=document.getElementById('audience')?document.getElementById('audience').value:'clinician';
1047
+ fetch(getApi()+'/chat/stream',{
1048
+ method:'POST', headers:{'Content-Type':'application/json'},
1049
+ body:JSON.stringify({
1050
+ messages:mammoChatHistory, audience:aud,
1051
+ prediction:mammoResult.prediction||'',
1052
+ confidence:mammoResult.confidence||0,
1053
+ logits:mammoResult.logits||[0,0],
1054
+ birads:mammoResult.birads||'',
1055
+ spatial_summary:mammoResult.spatial_summary||mammoResult.detail||''
1056
+ })
1057
+ }).then(function(r){
1058
+ if(!r.ok||!r.body){ bubble.textContent='Error: '+(r.statusText||'no response'); return; }
1059
+ var reader=r.body.getReader(), dec=new TextDecoder(), acc=''; bubble.textContent='';
1060
+ function pump(){ return reader.read().then(function(res){
1061
+ if(res.done){ mammoChatHistory.push({role:'assistant',content:acc}); return; }
1062
+ acc+=dec.decode(res.value,{stream:true}); bubble.innerHTML=fmtMd(acc);
1063
+ var b=document.getElementById('mammochatbody'); b.scrollTop=b.scrollHeight; return pump();
1064
+ });}
1065
+ return pump();
1066
+ }).catch(function(e){ bubble.textContent='Error: '+e.message; });
1067
+ }
1068
+
1069
+ addAMsg("Hello, I'm your AI radiology assistant. I can explain the model, training methodology, Grad-CAM, and more. What would you like to know?",'ai');
1070
+ checkHealth();
1071
+ </script>
1072
+ </body>
1073
+ </html>
frontend/samples/histo-normal.png ADDED

Git LFS Details

  • SHA256: 5e737ab62a22fa20af0af16a20bc563a39aa5e2fd7e649db5179e3b3c1f09473
  • Pointer size: 130 Bytes
  • Size of remote file: 26.2 kB
frontend/samples/histo-tumor.png ADDED

Git LFS Details

  • SHA256: d838a40bff54b826d2e040a5db4fd3586b96baaa17370491df78f481b896e419
  • Pointer size: 130 Bytes
  • Size of remote file: 25.3 kB
frontend/samples/mammo-benign.png ADDED

Git LFS Details

  • SHA256: 2d7aa16a2d356019f26fcd50a47cba3fd7dc3c8f954328a78ac2e3b096ffcd9d
  • Pointer size: 130 Bytes
  • Size of remote file: 71.4 kB
frontend/samples/mammo-cancer.png ADDED

Git LFS Details

  • SHA256: d93e2d9926d025184ed91ffa4a04a1d9e8a285ae25b1191dbf58a790a59a66b2
  • Pointer size: 131 Bytes
  • Size of remote file: 172 kB
modal_mammogram.py ADDED
@@ -0,0 +1,858 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ modal_mammogram.py
3
+ ───────────────────
4
+ Research-grade EfficientNet-B4 mammogram training β€” five innovations:
5
+
6
+ 1. Multi-View Patient Fusion β€” Siamese EfficientNet + ViewAttentionFusion
7
+ 2. Focal Loss β€” Ξ±=0.25, Ξ³=2, handles 1.5% cancer rate
8
+ 3. Mixed Precision (AMP) β€” 2Γ— faster, GradScaler
9
+ 4. Test-Time Augmentation β€” in mammogram_inference.py
10
+ 5. Progressive Resizing β€” 256β†’384β†’512 curriculum
11
+
12
+ Datasets:
13
+ Training: RSNA 2022 (USA, Kaggle) 54,706 images
14
+ External val: VinDr-Mammo (Vietnam, Kaggle) 20,000 images
15
+
16
+ Run:
17
+ modal run modal_mammogram.py # full training
18
+ modal run modal_mammogram.py --debug # 5%, 2 epochs
19
+ modal run modal_mammogram.py --skip-phase1 # resume phase 2
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ import csv
25
+ import json
26
+ import logging
27
+ import os
28
+ import time
29
+ from pathlib import Path
30
+
31
+ import modal
32
+
33
+ # ── Modal app ──────────────────────────────────────────────────────────────────
34
+ app = modal.App("medai-mammogram-research")
35
+
36
+ image = (
37
+ modal.Image.debian_slim(python_version="3.11")
38
+ .pip_install(
39
+ "torch==2.2.0", "torchvision==0.17.0",
40
+ "numpy<2.0", "pillow",
41
+ "pydicom", "pylibjpeg", "python-gdcm",
42
+ "kaggle", "pandas", "scikit-learn",
43
+ "scipy", "tqdm", "matplotlib",
44
+ )
45
+ .apt_install("libgomp1", "unzip", "wget")
46
+
47
+ )
48
+
49
+ rsna_vol = modal.Volume.from_name("rsna-mammogram-cache", create_if_missing=True)
50
+ vindr_vol = modal.Volume.from_name("vindr-mammo-cache", create_if_missing=True)
51
+ out_vol = modal.Volume.from_name("mammogram-outputs", create_if_missing=True)
52
+
53
+ DATA_RSNA = "/data/rsna"
54
+ DATA_VINDR = "/data/vindr"
55
+ OUT_DIR = "/outputs"
56
+
57
+ logging.basicConfig(
58
+ level=logging.INFO,
59
+ format="%(asctime)s %(levelname)-8s %(message)s",
60
+ datefmt="%H:%M:%S",
61
+ )
62
+ logger = logging.getLogger(__name__)
63
+
64
+
65
+ # ── DICOM loader ───────────────────────────────────────────────────────────────
66
+ def _dicom_to_rgb(path: str):
67
+ import numpy as np, pydicom
68
+ from PIL import Image
69
+ from pydicom.pixel_data_handlers.util import apply_voi_lut
70
+ dcm = pydicom.dcmread(path)
71
+ try:
72
+ arr = apply_voi_lut(dcm.pixel_array, dcm)
73
+ except Exception:
74
+ arr = dcm.pixel_array.astype(np.float32)
75
+ arr = arr.astype(np.float32)
76
+ mn, mx = arr.min(), arr.max()
77
+ if mx > mn:
78
+ arr = (arr - mn) / (mx - mn) * 255.0
79
+ arr = arr.astype(np.uint8)
80
+ if getattr(dcm, "PhotometricInterpretation", "") == "MONOCHROME1":
81
+ arr = 255 - arr
82
+ if arr.ndim == 2:
83
+ arr = np.stack([arr, arr, arr], axis=-1)
84
+ return Image.fromarray(arr, mode="RGB")
85
+
86
+
87
+ # ── Metrics ────────────────────────────────────────────────────────────────────
88
+ def compute_metrics(y_true, y_prob, threshold=0.5):
89
+ import numpy as np
90
+ from sklearn.metrics import roc_auc_score, confusion_matrix, f1_score
91
+ y_true = np.array(y_true)
92
+ y_prob = np.array(y_prob)
93
+ y_pred = (y_prob >= threshold).astype(int)
94
+ auc = roc_auc_score(y_true, y_prob) if len(set(y_true)) > 1 else 0.0
95
+ cm = confusion_matrix(y_true, y_pred)
96
+ tn, fp, fn, tp = cm.ravel() if cm.size == 4 else (0, 0, 0, 0)
97
+ return {
98
+ "auc": round(float(auc), 4),
99
+ "sensitivity": round(tp / max(tp + fn, 1), 4),
100
+ "specificity": round(tn / max(tn + fp, 1), 4),
101
+ "ppv": round(tp / max(tp + fp, 1), 4),
102
+ "npv": round(tn / max(tn + fn, 1), 4),
103
+ "accuracy": round((tp + tn) / max(len(y_true), 1), 4),
104
+ "f1": round(float(f1_score(y_true, y_pred, zero_division=0)), 4),
105
+ "n_pos": int(y_true.sum()),
106
+ "n_neg": int((1 - y_true).sum()),
107
+ }
108
+
109
+
110
+ def youden_threshold(y_true, y_prob):
111
+ import numpy as np
112
+ from sklearn.metrics import roc_curve
113
+ fpr, tpr, thr = roc_curve(y_true, y_prob)
114
+ j = tpr - fpr
115
+ return float(thr[np.argmax(j)])
116
+
117
+
118
+ def bootstrap_auc_ci(y_true, y_prob, n=1000):
119
+ import numpy as np
120
+ from sklearn.metrics import roc_auc_score
121
+ rng = np.random.default_rng(42)
122
+ y_true, y_prob = np.array(y_true), np.array(y_prob)
123
+ aucs = []
124
+ for _ in range(n):
125
+ idx = rng.integers(0, len(y_true), len(y_true))
126
+ yt, yp = y_true[idx], y_prob[idx]
127
+ if len(set(yt)) > 1:
128
+ aucs.append(roc_auc_score(yt, yp))
129
+ return round(float(np.percentile(aucs, 2.5)), 4), round(float(np.percentile(aucs, 97.5)), 4)
130
+
131
+
132
+ # ── Training function ──────────────────���───────────────────────────────────────
133
+ @app.function(
134
+ image=image, gpu="a10g", timeout=86400,
135
+ volumes={DATA_RSNA: rsna_vol, DATA_VINDR: vindr_vol, OUT_DIR: out_vol},
136
+ secrets=[modal.Secret.from_name("kaggle"), modal.Secret.from_name("physionet")],
137
+
138
+ )
139
+ def train(
140
+ phase1_epochs: int = 5,
141
+ phase2_epochs: int = 15,
142
+ batch_size: int = 8,
143
+ phase1_lr: float = 3e-4,
144
+ phase2_lr: float = 5e-5,
145
+ max_lr: float = 1e-3,
146
+ focal_alpha: float = 0.25,
147
+ focal_gamma: float = 2.0,
148
+ skip_phase1: bool = False,
149
+ debug: bool = False,
150
+ seed: int = 42,
151
+ num_workers: int = 4,
152
+ ) -> dict:
153
+
154
+ import random, sys
155
+ import numpy as np
156
+ import pandas as pd
157
+ import torch
158
+ import torch.nn as nn
159
+ from PIL import Image
160
+ from sklearn.model_selection import train_test_split
161
+ from torch.optim import AdamW
162
+ from torch.optim.lr_scheduler import OneCycleLR
163
+ from torch.utils.data import Dataset, DataLoader
164
+ from torchvision import transforms
165
+
166
+ os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
167
+ random.seed(seed)
168
+ np.random.seed(seed)
169
+ torch.manual_seed(seed)
170
+ torch.cuda.manual_seed_all(seed)
171
+
172
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
173
+ if torch.cuda.is_available():
174
+ logger.info("GPU : %s", torch.cuda.get_device_name(0))
175
+ logger.info("VRAM: %.1f GB",
176
+ torch.cuda.get_device_properties(0).total_memory / 1e9)
177
+
178
+ logger.info("=" * 70)
179
+ logger.info(" EfficientNet-B4 Mammogram Training β€” 5 Innovations")
180
+ logger.info(" 1. Multi-view patient fusion (Siamese + Attention)")
181
+ logger.info(" 2. Focal Loss (Ξ±=%.2f, Ξ³=%.1f)",
182
+ focal_alpha, focal_gamma)
183
+ logger.info(" 3. Mixed Precision AMP (GradScaler)")
184
+ logger.info(" 4. TTA (inference β€” mammogram_inference.py)")
185
+ logger.info(" 5. Progressive Resizing (256β†’384β†’512)")
186
+ logger.info(" Train: RSNA 2022 (USA)")
187
+ logger.info(" Ext val: VinDr-Mammo (Vietnam)")
188
+ logger.info("=" * 70)
189
+
190
+ # ── Download datasets ──────────────────────────────────────────────────────
191
+ import subprocess
192
+
193
+ rsna_dir = Path(DATA_RSNA)
194
+ vindr_dir = Path(DATA_VINDR)
195
+
196
+ if not (rsna_dir / "train.csv").exists():
197
+ logger.info("Downloading RSNA 2022 (~300 GB)...")
198
+ os.makedirs(str(rsna_dir), exist_ok=True)
199
+
200
+ # Delete any partial zip on the volume from previous failed run
201
+ for old_zip in [rsna_dir / "rsna-breast-cancer-detection.zip"]:
202
+ if old_zip.exists():
203
+ logger.info("Removing partial zip from previous run...")
204
+ os.remove(str(old_zip))
205
+
206
+ # Download zip to /tmp (ephemeral container storage, not the volume)
207
+ # Volume only stores unzipped files (~300GB), not zip+unzipped (~570GB)
208
+ logger.info("Downloading zip to /tmp...")
209
+ subprocess.run([
210
+ "kaggle", "competitions", "download",
211
+ "-c", "rsna-breast-cancer-detection",
212
+ "-p", "/tmp",
213
+ ], check=True)
214
+
215
+ import glob as _glob
216
+ zips = _glob.glob("/tmp/*.zip")
217
+ tmp_zip = zips[0] if zips else "/tmp/rsna-breast-cancer-detection.zip"
218
+
219
+ logger.info("Unzipping to volume (%s)...", rsna_dir)
220
+ subprocess.run(["unzip", "-q", tmp_zip, "-d", str(rsna_dir)], check=True)
221
+
222
+ os.remove(tmp_zip)
223
+ logger.info("Done β€” zip removed from /tmp.")
224
+ rsna_vol.commit()
225
+
226
+ if not (vindr_dir / "breast-level_annotations.csv").exists():
227
+ logger.info("Downloading VinDr-Mammo from PhysioNet (~70 GB)...")
228
+ os.makedirs(str(vindr_dir), exist_ok=True)
229
+ pn_user = os.environ.get("PHYSIONET_USERNAME", "")
230
+ pn_pass = os.environ.get("PHYSIONET_PASSWORD", "")
231
+ # Download only CSV annotations first (fast, ~1MB)
232
+ for csv_file in ["breast-level_annotations.csv", "finding_annotations.csv"]:
233
+ subprocess.run([
234
+ "wget", "-N", "-c", "-q",
235
+ f"--user={pn_user}",
236
+ f"--password={pn_pass}",
237
+ "-P", str(vindr_dir),
238
+ f"https://physionet.org/files/vindr-mammo/1.0.0/{csv_file}",
239
+ ], check=True)
240
+
241
+ # Download DICOM images recursively β€” skip HTML pages
242
+ subprocess.run([
243
+ "wget", "-r", "-N", "-c", "-np", "-q",
244
+ "--accept=*.dicom,*.dcm",
245
+ "--reject=*.html,*.php,index*,robots*",
246
+ f"--user={pn_user}",
247
+ f"--password={pn_pass}",
248
+ "-P", str(vindr_dir),
249
+ "https://physionet.org/files/vindr-mammo/1.0.0/images/",
250
+ ], check=True)
251
+ import shutil
252
+ nested = vindr_dir / "physionet.org" / "files" / "vindr-mammo" / "1.0.0"
253
+ if nested.exists():
254
+ for item in nested.iterdir():
255
+ shutil.move(str(item), str(vindr_dir / item.name))
256
+ shutil.rmtree(str(vindr_dir / "physionet.org"))
257
+ vindr_vol.commit()
258
+ logger.info("VinDr-Mammo downloaded from PhysioNet.")
259
+
260
+ # ── Load and prepare RSNA labels ───────────────────────────────────────────
261
+ rsna_df = pd.read_csv(rsna_dir / "train.csv")
262
+ logger.info("RSNA β€” %d images | cancer rate: %.1f%%",
263
+ len(rsna_df), 100 * rsna_df["cancer"].mean())
264
+
265
+ if debug:
266
+ rsna_df = rsna_df.sample(frac=0.05, random_state=seed).reset_index(drop=True)
267
+ phase1_epochs = min(2, phase1_epochs)
268
+ phase2_epochs = min(2, phase2_epochs)
269
+ logger.info("DEBUG: %d RSNA samples", len(rsna_df))
270
+
271
+ # ── Innovation 1: Build multi-view pairs ───────────────────────────────────
272
+ # Group by patient + laterality to get CC+MLO pairs
273
+ # Each case = one breast = (CC image, MLO image, cancer label)
274
+ logger.info("Building multi-view pairs (patient Γ— laterality)...")
275
+ cases = []
276
+ view_col = "view" if "view" in rsna_df.columns else None
277
+
278
+ if view_col and "laterality" in rsna_df.columns:
279
+ for (pid, lat), grp in rsna_df.groupby(["patient_id", "laterality"]):
280
+ cc_rows = grp[grp[view_col] == "CC"]
281
+ mlo_rows = grp[grp[view_col] == "MLO"]
282
+ label = int(grp["cancer"].max())
283
+ if len(cc_rows) > 0 and len(mlo_rows) > 0:
284
+ cases.append({
285
+ "patient_id": pid,
286
+ "laterality": lat,
287
+ "cc_img": cc_rows.iloc[0]["image_id"],
288
+ "mlo_img": mlo_rows.iloc[0]["image_id"],
289
+ "label": label,
290
+ "has_pair": True,
291
+ })
292
+ else:
293
+ # Single view fallback β€” duplicate for both inputs
294
+ any_row = grp.iloc[0]
295
+ cases.append({
296
+ "patient_id": pid,
297
+ "laterality": lat,
298
+ "cc_img": any_row["image_id"],
299
+ "mlo_img": any_row["image_id"],
300
+ "label": label,
301
+ "has_pair": False,
302
+ })
303
+ else:
304
+ # Dataset doesn't have view column β€” fall back to single-view
305
+ logger.warning("No 'view' column found β€” using single-view mode.")
306
+ for _, row in rsna_df.iterrows():
307
+ cases.append({
308
+ "patient_id": row["patient_id"],
309
+ "laterality": row.get("laterality", "L"),
310
+ "cc_img": row["image_id"],
311
+ "mlo_img": row["image_id"],
312
+ "label": int(row["cancer"]),
313
+ "has_pair": False,
314
+ })
315
+
316
+ cases_df = pd.DataFrame(cases)
317
+ paired = cases_df["has_pair"].sum()
318
+ logger.info("Multi-view pairs built: %d total (%d with CC+MLO, %d single-view)",
319
+ len(cases_df), paired, len(cases_df) - paired)
320
+
321
+ train_cases, val_cases = train_test_split(
322
+ cases_df, test_size=0.15,
323
+ stratify=cases_df["label"], random_state=seed,
324
+ )
325
+ logger.info("Train cases: %d | Val cases: %d",
326
+ len(train_cases), len(val_cases))
327
+
328
+ # ── Load VinDr external validation ────────────────────────────────────────
329
+ vindr_csv = vindr_dir / "breast-level_annotations.csv"
330
+ vindr_df = pd.read_csv(vindr_csv)
331
+ bc = "breast_birads" if "breast_birads" in vindr_df.columns else "birads"
332
+ vindr_df["label"] = vindr_df[bc].map({
333
+ "BI-RADS 1": 0, "BI-RADS 2": 0,
334
+ "BI-RADS 4": 1, "BI-RADS 5": 1,
335
+ 1: 0, 2: 0, 4: 1, 5: 1,
336
+ })
337
+ vindr_df = vindr_df.dropna(subset=["label"])
338
+ split_col = "split" if "split" in vindr_df.columns else None
339
+ if split_col:
340
+ vindr_ext = vindr_df[vindr_df[split_col] == "test"].reset_index(drop=True)
341
+ else:
342
+ _, vindr_ext = train_test_split(
343
+ vindr_df, test_size=0.2, stratify=vindr_df["label"], random_state=seed
344
+ )
345
+ logger.info("VinDr external val: %d images | cancer: %.1f%%",
346
+ len(vindr_ext), 100 * vindr_ext["label"].mean())
347
+
348
+ # ── Innovation 5: Progressive resizing transforms ──────────────────────────
349
+ IMGNET = ([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
350
+
351
+ def make_train_tf(size):
352
+ return transforms.Compose([
353
+ transforms.Resize((size, size)),
354
+ transforms.RandomHorizontalFlip(),
355
+ transforms.RandomVerticalFlip(p=0.2),
356
+ transforms.RandomRotation(10),
357
+ transforms.ColorJitter(brightness=0.15, contrast=0.15),
358
+ transforms.RandomAffine(0, translate=(0.05, 0.05), scale=(0.95, 1.05)),
359
+ transforms.ToTensor(),
360
+ transforms.Normalize(*IMGNET),
361
+ ])
362
+
363
+ def make_val_tf(size=512):
364
+ return transforms.Compose([
365
+ transforms.Resize((size, size)),
366
+ transforms.ToTensor(),
367
+ transforms.Normalize(*IMGNET),
368
+ ])
369
+
370
+ # Progressive resizing schedule:
371
+ # Phase 1 (epochs 1-5): 256Γ—256
372
+ # Phase 2 first half: 384Γ—384
373
+ # Phase 2 second half: 512Γ—512
374
+ def get_size_for_epoch(epoch, p1_epochs, p2_epochs, is_phase2):
375
+ if not is_phase2:
376
+ return 256
377
+ midpoint = p2_epochs // 2
378
+ if epoch <= midpoint:
379
+ return 384
380
+ return 512
381
+
382
+ # ── Dataset classes ────────────────────────────────────────────────────────
383
+ class MultiViewDataset(Dataset):
384
+ def __init__(self, cases_df, img_dir, transform, size):
385
+ self.cases = cases_df.reset_index(drop=True)
386
+ self.img_dir = Path(img_dir)
387
+ self.transform = transform
388
+ self.size = size
389
+
390
+ def _load(self, patient_id, image_id):
391
+ path = self.img_dir / "train_images" / str(patient_id) / f"{image_id}.dcm"
392
+ try:
393
+ return _dicom_to_rgb(str(path))
394
+ except Exception as e:
395
+ logger.warning("Load error %s: %s", path.name, e)
396
+ return Image.new("RGB", (self.size, self.size), 0)
397
+
398
+ def __len__(self):
399
+ return len(self.cases)
400
+
401
+ def __getitem__(self, idx):
402
+ row = self.cases.iloc[idx]
403
+ cc_img = self._load(row["patient_id"], row["cc_img"])
404
+ mlo_img = self._load(row["patient_id"], row["mlo_img"])
405
+ label = int(row["label"])
406
+ return (
407
+ self.transform(cc_img),
408
+ self.transform(mlo_img),
409
+ label,
410
+ )
411
+
412
+ class VinDrDataset(Dataset):
413
+ def __init__(self, df, img_dir, transform):
414
+ self.df = df.reset_index(drop=True)
415
+ self.img_dir = Path(img_dir)
416
+ self.transform = transform
417
+
418
+ def __len__(self):
419
+ return len(self.df)
420
+
421
+ def __getitem__(self, idx):
422
+ row = self.df.iloc[idx]
423
+ study = str(row.get("study_id", ""))
424
+ img_id = str(row.get("image_id", ""))
425
+ label = int(row["label"])
426
+ for ext in [".dicom", ".dcm"]:
427
+ path = self.img_dir / "images" / study / f"{img_id}{ext}"
428
+ if path.exists():
429
+ break
430
+ try:
431
+ img = _dicom_to_rgb(str(path))
432
+ except Exception:
433
+ img = Image.new("RGB", (512, 512), 0)
434
+ return self.transform(img), label
435
+
436
+ val_tf_512 = make_val_tf(512)
437
+
438
+ # ── Class weight and Focal Loss ────────────────────────────────────────────
439
+ n_pos = int(train_cases["label"].sum())
440
+ n_neg = len(train_cases) - n_pos
441
+ pos_weight = torch.tensor([n_neg / max(n_pos, 1)], device=device)
442
+ logger.info("Class weight: %.1f (cancer: %.1f%%)",
443
+ pos_weight.item(), 100 * n_pos / len(train_cases))
444
+
445
+ # ── Inline model definitions (no file mounting needed) ──────────────────
446
+ import torch.nn as nn
447
+ import torch.nn.functional as F
448
+ from torchvision import models
449
+ from torchvision.models import EfficientNet_B4_Weights
450
+
451
+ class FocalLoss(nn.Module):
452
+ def __init__(self, alpha=0.25, gamma=2.0, pos_weight=None, label_smoothing=0.1):
453
+ super().__init__()
454
+ self.alpha = alpha; self.gamma = gamma
455
+ self.pos_weight = pos_weight; self.label_smoothing = label_smoothing
456
+ def forward(self, logits, targets):
457
+ ce = F.cross_entropy(logits, targets, weight=self.pos_weight,
458
+ label_smoothing=self.label_smoothing, reduction="none")
459
+ pt = torch.exp(-ce)
460
+ return (self.alpha * (1 - pt) ** self.gamma * ce).mean()
461
+
462
+ class ViewAttentionFusion(nn.Module):
463
+ def __init__(self, dim=1792):
464
+ super().__init__()
465
+ self.gate = nn.Sequential(
466
+ nn.Linear(dim*2, dim//4), nn.ReLU(inplace=True),
467
+ nn.Dropout(0.2), nn.Linear(dim//4, 2))
468
+ self.residual_w = nn.Parameter(torch.tensor(0.5))
469
+ def forward(self, cc, mlo):
470
+ w = torch.softmax(self.gate(torch.cat([cc, mlo], -1)), -1)
471
+ att = w[:,0:1]*cc + w[:,1:2]*mlo
472
+ s = torch.sigmoid(self.residual_w)
473
+ return s*att + (1-s)*0.5*(cc+mlo)
474
+
475
+ class MultiViewMammogramClassifier(nn.Module):
476
+ def __init__(self, pretrained=True, freeze_backbone=False, dropout_rate=0.4):
477
+ super().__init__()
478
+ weights = EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None
479
+ bb = models.efficientnet_b4(weights=weights)
480
+ self.features = bb.features; self.avgpool = bb.avgpool
481
+ if freeze_backbone:
482
+ for p in self.features.parameters(): p.requires_grad = False
483
+ self.fusion = ViewAttentionFusion(1792)
484
+ self.classifier = nn.Sequential(
485
+ nn.BatchNorm1d(1792), nn.Dropout(dropout_rate),
486
+ nn.Linear(1792, 512), nn.ReLU(inplace=True),
487
+ nn.BatchNorm1d(512), nn.Dropout(dropout_rate*0.75),
488
+ nn.Linear(512, 2))
489
+ def encode(self, x):
490
+ return torch.flatten(self.avgpool(self.features(x)), 1)
491
+ def forward(self, cc, mlo):
492
+ logits = self.classifier(self.fusion(self.encode(cc), self.encode(mlo)))
493
+ return {"logits": logits, "probs": torch.softmax(logits, 1)}
494
+ def forward_single(self, x):
495
+ return self.forward(x, x)
496
+
497
+ # Innovation 2: Focal Loss
498
+ def make_criterion():
499
+ return FocalLoss(
500
+ alpha = focal_alpha,
501
+ gamma = focal_gamma,
502
+ pos_weight = torch.tensor(
503
+ [1.0, pos_weight.item()], device=device
504
+ ),
505
+ label_smoothing = 0.1,
506
+ )
507
+
508
+ # ── Training epoch (AMP + multi-view) ─────────────────────────────────────
509
+ def run_train_epoch(model, loader, criterion, optimizer, scheduler, scaler):
510
+ """Innovation 3: AMP mixed precision training."""
511
+ model.train()
512
+ total_loss = tp = total = pos = 0
513
+ with torch.enable_grad():
514
+ for cc_imgs, mlo_imgs, labels in loader:
515
+ cc_imgs = cc_imgs.to(device, non_blocking=True)
516
+ mlo_imgs = mlo_imgs.to(device, non_blocking=True)
517
+ labels = labels.long().to(device, non_blocking=True)
518
+
519
+ # AMP autocast
520
+ with torch.autocast(device_type="cuda", dtype=torch.float16):
521
+ out = model(cc_imgs, mlo_imgs)
522
+ loss = criterion(out["logits"], labels)
523
+
524
+ optimizer.zero_grad()
525
+ scaler.scale(loss).backward()
526
+ scaler.unscale_(optimizer)
527
+ nn.utils.clip_grad_norm_(model.parameters(), 1.0)
528
+ scaler.step(optimizer)
529
+ scaler.update()
530
+ scheduler.step()
531
+
532
+ preds = out["logits"].argmax(1)
533
+ total += labels.size(0)
534
+ total_loss += loss.item() * labels.size(0)
535
+ mask = (labels == 1)
536
+ tp += (preds[mask] == 1).sum().item()
537
+ pos += mask.sum().item()
538
+
539
+ return total_loss / max(total, 1), tp / max(pos, 1)
540
+
541
+ def run_eval_epoch(model, loader, is_multiview=True):
542
+ """Eval on multi-view or single-view loader."""
543
+ model.eval()
544
+ all_probs, all_labels = [], []
545
+ with torch.inference_mode():
546
+ for batch in loader:
547
+ if is_multiview:
548
+ cc_imgs, mlo_imgs, labels = batch
549
+ cc_imgs = cc_imgs.to(device)
550
+ mlo_imgs = mlo_imgs.to(device)
551
+ out = model(cc_imgs, mlo_imgs)
552
+ else:
553
+ imgs, labels = batch
554
+ imgs = imgs.to(device)
555
+ out = model.forward_single(imgs)
556
+ probs = out["probs"][:, 1].cpu().numpy()
557
+ all_probs.extend(probs.tolist())
558
+ all_labels.extend(labels.numpy().tolist())
559
+ return all_labels, all_probs
560
+
561
+ def save_checkpoint(model, optimizer, epoch, metrics, path):
562
+ torch.save({
563
+ "epoch": epoch,
564
+ "state_dict": model.state_dict(),
565
+ "optimizer": optimizer.state_dict(),
566
+ "metrics": metrics,
567
+ }, path)
568
+
569
+ # ── Phase 1: Frozen backbone, 256Γ—256 ─────────────────────────────────────
570
+ if not skip_phase1:
571
+ logger.info("\n" + "─" * 70)
572
+ logger.info(" PHASE 1 β€” Head only, 256Γ—256 (%d epochs)", phase1_epochs)
573
+ logger.info("─" * 70)
574
+
575
+ model = MultiViewMammogramClassifier(
576
+ pretrained=True, freeze_backbone=True
577
+ ).to(device)
578
+ criterion = make_criterion()
579
+ optimizer = AdamW(
580
+ [p for p in model.parameters() if p.requires_grad],
581
+ lr=phase1_lr, weight_decay=1e-4,
582
+ )
583
+ scaler = torch.cuda.amp.GradScaler()
584
+
585
+ train_ds = MultiViewDataset(train_cases, rsna_dir, make_train_tf(256), 256)
586
+ val_ds = MultiViewDataset(val_cases, rsna_dir, val_tf_512, 512)
587
+ train_loader = DataLoader(train_ds, batch_size, True, num_workers=num_workers, pin_memory=True)
588
+ val_loader = DataLoader(val_ds, batch_size, False, num_workers=num_workers, pin_memory=True)
589
+
590
+ scheduler = OneCycleLR(
591
+ optimizer, max_lr=phase1_lr * 3,
592
+ steps_per_epoch=len(train_loader), epochs=phase1_epochs, pct_start=0.3,
593
+ )
594
+
595
+ best_auc = 0.0
596
+ for epoch in range(1, phase1_epochs + 1):
597
+ t0 = time.time()
598
+ tr_loss, tr_sens = run_train_epoch(
599
+ model, train_loader, criterion, optimizer, scheduler, scaler
600
+ )
601
+ vl_labels, vl_probs = run_eval_epoch(model, val_loader)
602
+ vl_m = compute_metrics(vl_labels, vl_probs)
603
+ logger.info(
604
+ "P1 E%02d | 256px | loss=%.4f tr_sens=%.3f | RSNA AUC=%.4f sens=%.3f | %.0fs",
605
+ epoch, tr_loss, tr_sens,
606
+ vl_m["auc"], vl_m["sensitivity"], time.time() - t0,
607
+ )
608
+ if vl_m["auc"] > best_auc:
609
+ best_auc = vl_m["auc"]
610
+ save_checkpoint(model, optimizer, epoch, vl_m,
611
+ f"{OUT_DIR}/mammogram_phase1.pth")
612
+ logger.info(" βœ“ Phase 1 checkpoint (AUC=%.4f)", best_auc)
613
+ out_vol.commit()
614
+ logger.info("Phase 1 complete. Best AUC: %.4f", best_auc)
615
+
616
+ # ── Phase 2: Full fine-tuning, progressive resizing ───────────────────────
617
+ logger.info("\n" + "─" * 70)
618
+ logger.info(" PHASE 2 β€” Full fine-tuning, progressive resizing (%d epochs)",
619
+ phase2_epochs)
620
+ logger.info(" Epochs 1–%-2d : 256Γ—256", phase2_epochs // 3)
621
+ logger.info(" Epochs %-2d–%-2d : 384Γ—384",
622
+ phase2_epochs // 3 + 1, phase2_epochs * 2 // 3)
623
+ logger.info(" Epochs %-2d–%-2d : 512Γ—512",
624
+ phase2_epochs * 2 // 3 + 1, phase2_epochs)
625
+ logger.info("─" * 70)
626
+
627
+ model = MultiViewMammogramClassifier(pretrained=False, freeze_backbone=False).to(device)
628
+ phase1_ckpt = f"{OUT_DIR}/mammogram_phase1.pth"
629
+ if Path(phase1_ckpt).exists():
630
+ ckpt = torch.load(phase1_ckpt, map_location=device)
631
+ model.load_state_dict(ckpt["state_dict"])
632
+ logger.info("Loaded Phase 1 checkpoint (AUC=%.4f)",
633
+ ckpt["metrics"].get("auc", 0))
634
+
635
+ criterion = make_criterion()
636
+ optimizer = AdamW(model.parameters(), lr=phase2_lr, weight_decay=1e-4)
637
+ scaler = torch.cuda.amp.GradScaler()
638
+
639
+ # Build full train/val loaders at 512 (size updated per epoch below)
640
+ val_ds_512 = MultiViewDataset(val_cases, rsna_dir, val_tf_512, 512)
641
+ vindr_ds = VinDrDataset(vindr_ext, vindr_dir, make_val_tf(512))
642
+ val_loader = DataLoader(val_ds_512, batch_size, False, num_workers=num_workers, pin_memory=True)
643
+ vindr_loader = DataLoader(vindr_ds, batch_size, False, num_workers=num_workers, pin_memory=True)
644
+
645
+ # Scheduler needs total steps β€” compute with 512 loader
646
+ train_ds_512 = MultiViewDataset(train_cases, rsna_dir, make_train_tf(512), 512)
647
+ tmp_loader = DataLoader(train_ds_512, batch_size, True, num_workers=num_workers)
648
+ steps_per_epoch = len(tmp_loader)
649
+ del tmp_loader, train_ds_512
650
+
651
+ scheduler = OneCycleLR(
652
+ optimizer, max_lr=max_lr,
653
+ steps_per_epoch=steps_per_epoch,
654
+ epochs=phase2_epochs, pct_start=0.3,
655
+ )
656
+
657
+ best_auc = 0.0
658
+ best_epoch = 0
659
+ best_thr = 0.5
660
+ log_rows = []
661
+
662
+ # Milestone epochs for progressive resizing
663
+ size_milestones = {
664
+ 1: 256,
665
+ phase2_epochs // 3 + 1: 384,
666
+ phase2_epochs * 2 // 3 + 1: 512,
667
+ }
668
+ current_size = 256
669
+ train_loader = DataLoader(
670
+ MultiViewDataset(train_cases, rsna_dir, make_train_tf(256), 256),
671
+ batch_size, True, num_workers=num_workers, pin_memory=True,
672
+ )
673
+
674
+ for epoch in range(1, phase2_epochs + 1):
675
+ t0 = time.time()
676
+
677
+ # Innovation 5: Progressive resizing β€” rebuild loader when size changes
678
+ if epoch in size_milestones:
679
+ new_size = size_milestones[epoch]
680
+ if new_size != current_size:
681
+ current_size = new_size
682
+ logger.info(" β†’ Resizing to %dΓ—%d (epoch %d)",
683
+ current_size, current_size, epoch)
684
+ train_loader = DataLoader(
685
+ MultiViewDataset(
686
+ train_cases, rsna_dir,
687
+ make_train_tf(current_size), current_size
688
+ ),
689
+ batch_size, True,
690
+ num_workers=num_workers, pin_memory=True,
691
+ )
692
+
693
+ tr_loss, tr_sens = run_train_epoch(
694
+ model, train_loader, criterion, optimizer, scheduler, scaler
695
+ )
696
+
697
+ # RSNA internal validation
698
+ rsna_labels, rsna_probs = run_eval_epoch(model, val_loader)
699
+ thr = youden_threshold(rsna_labels, rsna_probs)
700
+ rsna_m = compute_metrics(rsna_labels, rsna_probs, threshold=thr)
701
+
702
+ # VinDr external validation
703
+ vindr_labels, vindr_probs = run_eval_epoch(
704
+ model, vindr_loader, is_multiview=False
705
+ )
706
+ vindr_m = compute_metrics(vindr_labels, vindr_probs, threshold=thr)
707
+
708
+ elapsed = time.time() - t0
709
+
710
+ logger.info(
711
+ "E%02d/%d [%3dpx] β”‚ loss=%.4f sens=%.3f β”‚ "
712
+ "RSNA AUC=%.4f sens=%.3f spec=%.3f β”‚ "
713
+ "VinDr AUC=%.4f sens=%.3f spec=%.3f β”‚ %.0fs",
714
+ epoch, phase2_epochs, current_size, tr_loss, tr_sens,
715
+ rsna_m["auc"], rsna_m["sensitivity"], rsna_m["specificity"],
716
+ vindr_m["auc"], vindr_m["sensitivity"], vindr_m["specificity"],
717
+ elapsed,
718
+ )
719
+
720
+ log_rows.append({
721
+ "epoch": epoch, "size": current_size,
722
+ "train_loss": tr_loss, "train_sens": tr_sens,
723
+ "rsna_auc": rsna_m["auc"], "rsna_sens": rsna_m["sensitivity"],
724
+ "rsna_spec": rsna_m["specificity"], "rsna_ppv": rsna_m["ppv"],
725
+ "rsna_npv": rsna_m["npv"], "rsna_acc": rsna_m["accuracy"],
726
+ "vindr_auc": vindr_m["auc"], "vindr_sens": vindr_m["sensitivity"],
727
+ "vindr_spec": vindr_m["specificity"], "vindr_ppv": vindr_m["ppv"],
728
+ "vindr_npv": vindr_m["npv"], "vindr_acc": vindr_m["accuracy"],
729
+ "threshold": thr,
730
+ })
731
+
732
+ if rsna_m["auc"] > best_auc:
733
+ best_auc = rsna_m["auc"]
734
+ best_epoch = epoch
735
+ best_thr = thr
736
+ save_checkpoint(
737
+ model, optimizer, epoch,
738
+ {"rsna": rsna_m, "vindr": vindr_m, "threshold": thr},
739
+ f"{OUT_DIR}/mammogram_weights.pth",
740
+ )
741
+ logger.info(
742
+ " βœ“ Best checkpoint β€” RSNA AUC=%.4f VinDr AUC=%.4f [%dpx]",
743
+ rsna_m["auc"], vindr_m["auc"], current_size,
744
+ )
745
+ out_vol.commit()
746
+
747
+ # ── Final evaluation with bootstrap CI ────────────────────────────────────
748
+ logger.info("\n" + "=" * 70)
749
+ logger.info(" FINAL EVALUATION (epoch %d checkpoint)", best_epoch)
750
+ logger.info("=" * 70)
751
+
752
+ ckpt = torch.load(f"{OUT_DIR}/mammogram_weights.pth", map_location=device)
753
+ model.load_state_dict(ckpt["state_dict"])
754
+ model.eval()
755
+
756
+ rsna_l, rsna_p = run_eval_epoch(model, val_loader)
757
+ vindr_l, vindr_p = run_eval_epoch(model, vindr_loader, is_multiview=False)
758
+
759
+ rsna_f = compute_metrics(rsna_l, rsna_p, best_thr)
760
+ vindr_f = compute_metrics(vindr_l, vindr_p, best_thr)
761
+ rsna_ci = bootstrap_auc_ci(rsna_l, rsna_p)
762
+ vindr_ci = bootstrap_auc_ci(vindr_l, vindr_p)
763
+
764
+ gap = rsna_f["auc"] - vindr_f["auc"]
765
+
766
+ logger.info("\n RSNA 2022 (Internal validation β€” USA)")
767
+ logger.info(" AUC: %.4f (95%% CI: %.4f–%.4f)", rsna_f["auc"], *rsna_ci)
768
+ logger.info(" Sensitivity: %.1f%%", rsna_f["sensitivity"] * 100)
769
+ logger.info(" Specificity: %.1f%%", rsna_f["specificity"] * 100)
770
+ logger.info(" PPV: %.4f NPV: %.4f", rsna_f["ppv"], rsna_f["npv"])
771
+
772
+ logger.info("\n VinDr-Mammo (External validation β€” Vietnam) ← key publication metric")
773
+ logger.info(" AUC: %.4f (95%% CI: %.4f–%.4f)", vindr_f["auc"], *vindr_ci)
774
+ logger.info(" Sensitivity: %.1f%%", vindr_f["sensitivity"] * 100)
775
+ logger.info(" Specificity: %.1f%%", vindr_f["specificity"] * 100)
776
+ logger.info(" PPV: %.4f NPV: %.4f", vindr_f["ppv"], vindr_f["npv"])
777
+
778
+ logger.info("\n Generalisation gap: %.4f (%s)",
779
+ gap,
780
+ "βœ“ Excellent (<0.05)" if gap < 0.05 else
781
+ "⚠ Moderate (0.05–0.10)" if gap < 0.10 else
782
+ "βœ— Large β€” consider domain adaptation")
783
+
784
+ results = {
785
+ "best_epoch": best_epoch,
786
+ "threshold": best_thr,
787
+ "innovations": [
788
+ "Multi-view patient fusion (Siamese EfficientNet-B4 + ViewAttentionFusion)",
789
+ f"Focal Loss (Ξ±={focal_alpha}, Ξ³={focal_gamma})",
790
+ "Mixed Precision Training (AMP GradScaler)",
791
+ "Test-Time Augmentation (8 augments, in mammogram_inference.py)",
792
+ "Progressive Resizing (256β†’384β†’512)",
793
+ ],
794
+ "rsna_internal": {
795
+ **rsna_f, "auc_ci": rsna_ci,
796
+ },
797
+ "vindr_external": {
798
+ **vindr_f, "auc_ci": vindr_ci,
799
+ },
800
+ "generalisation_gap": round(gap, 4),
801
+ }
802
+
803
+ with open(f"{OUT_DIR}/mammogram_training_log.csv", "w", newline="") as f:
804
+ w = csv.DictWriter(f, fieldnames=log_rows[0].keys())
805
+ w.writeheader(); w.writerows(log_rows)
806
+
807
+ with open(f"{OUT_DIR}/mammogram_results.json", "w") as f:
808
+ json.dump(results, f, indent=2)
809
+
810
+ out_vol.commit()
811
+ logger.info("\n Saved: mammogram_weights.pth, training_log.csv, results.json")
812
+ logger.info("=" * 70)
813
+ return results
814
+
815
+
816
+ # ── Local entrypoint ───────────────────────────────────────────────────────────
817
+ @app.local_entrypoint()
818
+ def main(
819
+ phase1_epochs: int = 5,
820
+ phase2_epochs: int = 15,
821
+ batch_size: int = 8,
822
+ phase2_lr: float = 5e-5,
823
+ max_lr: float = 1e-3,
824
+ focal_alpha: float = 0.25,
825
+ focal_gamma: float = 2.0,
826
+ skip_phase1: bool = False,
827
+ debug: bool = False,
828
+ ):
829
+ print("=" * 70)
830
+ print(" MedAI β€” Research-Grade Mammogram Training")
831
+ print(" 5 Innovations:")
832
+ print(" 1. Multi-view patient fusion (Siamese + Attention)")
833
+ print(f" 2. Focal Loss (Ξ±={focal_alpha}, Ξ³={focal_gamma})")
834
+ print(" 3. Mixed Precision AMP")
835
+ print(" 4. Test-Time Augmentation (at inference)")
836
+ print(" 5. Progressive Resizing (256β†’384β†’512)")
837
+ print(f" Train: RSNA 2022 | External val: VinDr-Mammo")
838
+ print(f" Debug: {debug}")
839
+ print("=" * 70)
840
+
841
+ results = train.remote(
842
+ phase1_epochs=phase1_epochs, phase2_epochs=phase2_epochs,
843
+ batch_size=batch_size, phase2_lr=phase2_lr, max_lr=max_lr,
844
+ focal_alpha=focal_alpha, focal_gamma=focal_gamma,
845
+ skip_phase1=skip_phase1, debug=debug,
846
+ )
847
+
848
+ print("=" * 70)
849
+ print(" Job running independently on Modal GPU.")
850
+ print(" Your terminal and Mac can now be closed.")
851
+ print()
852
+ print(" Check progress:")
853
+ print(" modal app logs " + str(call.object_id) if hasattr(call, "object_id") else " modal.com/apps/relixsx/main")
854
+ print()
855
+ print(" When done, download weights:")
856
+ print(" modal volume get mammogram-outputs mammogram_weights.pth model/mammogram_weights.pth")
857
+ print(" modal volume get mammogram-outputs mammogram_results.json mammogram_results.json")
858
+ print("=" * 70)
model/__init__.py ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ from .inference import BreastCancerInferencePipeline
2
+ from .model import BreastCancerClassifier
3
+
4
+ __all__ = ["BreastCancerInferencePipeline", "BreastCancerClassifier"]
model/inference.py ADDED
@@ -0,0 +1,197 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ model/inference.py
3
+ ──────────────────
4
+ End-to-end inference pipeline.
5
+
6
+ Usage
7
+ ─────
8
+ from model import BreastCancerInferencePipeline
9
+
10
+ pipeline = BreastCancerInferencePipeline(weights_path="model/weights.pth")
11
+ result = pipeline.predict("slide_001.png")
12
+
13
+ # result β†’ {
14
+ # "prediction" : "malignant",
15
+ # "confidence" : 0.9341,
16
+ # "logits" : tensor([[-2.14, 3.87]])
17
+ # }
18
+
19
+ Output contract (per spec)
20
+ ──────────────────────────
21
+ {
22
+ "prediction" : str β€” "benign" | "malignant"
23
+ "confidence" : float β€” probability of the predicted class [0, 1]
24
+ "logits" : torch.Tensor β€” raw model outputs (1, 2), pre-softmax
25
+ }
26
+ """
27
+
28
+ from __future__ import annotations
29
+
30
+ import logging
31
+ from pathlib import Path
32
+ from typing import Optional, Union
33
+
34
+ import numpy as np
35
+ import torch
36
+ from PIL import Image
37
+
38
+ from .model import BreastCancerClassifier
39
+ from utils.preprocessing import ImagePreprocessor # utils/ is a sibling package
40
+
41
+ logger = logging.getLogger(__name__)
42
+
43
+ # Class index β†’ label mapping
44
+ LABEL_MAP: dict[int, str] = {0: "benign", 1: "malignant"}
45
+
46
+
47
+ class BreastCancerInferencePipeline:
48
+ """
49
+ Self-contained inference pipeline for breast cancer classification.
50
+
51
+ Parameters
52
+ ----------
53
+ weights_path : str | Path | None
54
+ Path to a saved state_dict (.pt / .pth). Conventionally stored at
55
+ model/weights.pth. If None, runs with ImageNet-pretrained backbone
56
+ weights only (useful for integration testing before fine-tuning).
57
+ device : str | None
58
+ "cuda", "mps", or "cpu". Auto-detected when None.
59
+ confidence_threshold : float
60
+ Minimum confidence to return the predicted label; below this,
61
+ the output still returns the argmax class but logs a low-confidence
62
+ warning. Default: 0.5.
63
+ """
64
+
65
+ def __init__(
66
+ self,
67
+ weights_path: Optional[Union[str, Path]] = None,
68
+ device: Optional[str] = None,
69
+ confidence_threshold: float = 0.5,
70
+ ) -> None:
71
+ self.device = self._resolve_device(device)
72
+ self.confidence_threshold = confidence_threshold
73
+ self.preprocessor = ImagePreprocessor()
74
+
75
+ # ── Build and load model ─────────────────────────────────────────────
76
+ self.model = BreastCancerClassifier(pretrained=(weights_path is None))
77
+ if weights_path is not None:
78
+ self._load_weights(weights_path)
79
+
80
+ self.model.eval()
81
+ self.model.to(self.device)
82
+ logger.info("Pipeline ready on device: %s", self.device)
83
+
84
+ # ────────────────────────────────────────────────────────────────────────
85
+ def predict(
86
+ self,
87
+ image: Union[str, Path, "Image.Image", np.ndarray, torch.Tensor],
88
+ ) -> dict:
89
+ """
90
+ Run end-to-end inference on a single histopathology image.
91
+
92
+ Parameters
93
+ ----------
94
+ image : str | Path | PIL.Image | np.ndarray | torch.Tensor
95
+ Raw image in any supported format (see utils/preprocessing.py).
96
+
97
+ Returns
98
+ -------
99
+ dict
100
+ {
101
+ "prediction" : str β€” "benign" or "malignant"
102
+ "confidence" : float β€” predicted-class probability [0, 1]
103
+ "logits" : Tensor[1,2] β€” raw pre-softmax scores
104
+ }
105
+ """
106
+ tensor = self._preprocess(image)
107
+
108
+ with torch.inference_mode():
109
+ output = self.model(tensor)
110
+
111
+ return self._format_output(output["logits"], output["probs"])
112
+
113
+ # ────────────────────────────────────────────────────────────────────────
114
+ def predict_batch(self, images: list) -> list[dict]:
115
+ """
116
+ Run inference on a batch of images.
117
+
118
+ Parameters
119
+ ----------
120
+ images : list of any supported image type
121
+
122
+ Returns
123
+ -------
124
+ list of prediction dicts (same schema as predict())
125
+ """
126
+ tensors = torch.cat([self._preprocess(img) for img in images], dim=0)
127
+ tensors = tensors.to(self.device)
128
+
129
+ with torch.inference_mode():
130
+ output = self.model(tensors)
131
+
132
+ results = []
133
+ for i in range(tensors.size(0)):
134
+ logit_i = output["logits"][i].unsqueeze(0) # (1, 2)
135
+ prob_i = output["probs"][i].unsqueeze(0) # (1, 2)
136
+ results.append(self._format_output(logit_i, prob_i))
137
+ return results
138
+
139
+ # ── Internal helpers ─────────────────────────────────────────────────────
140
+ def _preprocess(self, image) -> torch.Tensor:
141
+ """Preprocess to (1, 3, 224, 224) on the correct device."""
142
+ tensor = self.preprocessor(image) # (1, 3, 224, 224) CPU
143
+ return tensor.to(self.device)
144
+
145
+ def _format_output(
146
+ self,
147
+ logits: torch.Tensor,
148
+ probs: torch.Tensor,
149
+ ) -> dict:
150
+ """
151
+ Convert raw model tensors into the specced output dictionary.
152
+
153
+ Output contract
154
+ ───────────────
155
+ "prediction" : str β€” "benign" | "malignant"
156
+ "confidence" : float β€” probability of the predicted class
157
+ "logits" : Tensor[1, 2]
158
+ """
159
+ predicted_idx = int(torch.argmax(probs, dim=1).item())
160
+ confidence = float(probs[0, predicted_idx].item())
161
+ prediction = LABEL_MAP[predicted_idx]
162
+
163
+ if confidence < self.confidence_threshold:
164
+ logger.warning(
165
+ "Low-confidence prediction: %s (%.3f). "
166
+ "Treat result with caution.",
167
+ prediction, confidence,
168
+ )
169
+
170
+ return {
171
+ "prediction": prediction,
172
+ "confidence": round(confidence, 6),
173
+ "logits": logits.detach().cpu(), # (1, 2), kept for Grad-CAM
174
+ }
175
+
176
+ def _load_weights(self, path: Union[str, Path]) -> None:
177
+ """Load fine-tuned weights from a state_dict checkpoint."""
178
+ path = Path(path)
179
+ if not path.exists():
180
+ raise FileNotFoundError(f"Weights file not found: {path}")
181
+
182
+ checkpoint = torch.load(path, map_location=self.device)
183
+
184
+ # Support both raw state_dict and {'state_dict': ...} wrappers
185
+ state_dict = checkpoint.get("state_dict", checkpoint)
186
+ self.model.load_state_dict(state_dict, strict=True)
187
+ logger.info("Loaded weights from: %s", path)
188
+
189
+ @staticmethod
190
+ def _resolve_device(device: Optional[str]) -> torch.device:
191
+ if device is not None:
192
+ return torch.device(device)
193
+ if torch.cuda.is_available():
194
+ return torch.device("cuda")
195
+ if torch.backends.mps.is_available():
196
+ return torch.device("mps")
197
+ return torch.device("cpu")
model/mammogram_ensemble.py ADDED
@@ -0,0 +1,234 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ model/mammogram_ensemble.py
3
+ ────────────────────────────
4
+ Ensemble inference for the 3 EfficientNet-B4 mammogram models trained
5
+ on vast.ai (seeds 42, 123, 999).
6
+
7
+ IMPORTANT β€” architecture match
8
+ ───────────────────────────────
9
+ These weights were trained with the SIMPLE single-view architecture
10
+ (feat / pool / head), NOT the multi-view fusion model. This file
11
+ defines the exact matching architecture so the checkpoints load with
12
+ strict=True. Do not use MultiViewMammogramClassifier with these weights.
13
+
14
+ Ensemble result (RSNA internal validation):
15
+ Model seed=42: AUC 0.7989
16
+ Model seed=123: AUC 0.8254
17
+ Model seed=999: AUC 0.8083
18
+ ──────────────────────────────
19
+ Ensemble image AUC: 0.8436
20
+ Ensemble patient AUC: 0.8443 ← headline number
21
+ Sensitivity: 70.1% Specificity: 82.4%
22
+
23
+ Usage
24
+ ─────
25
+ from model.mammogram_ensemble import MammogramEnsemble
26
+ from PIL import Image
27
+
28
+ ens = MammogramEnsemble(
29
+ weight_paths=[
30
+ "model/model_s42.pth",
31
+ "model/model_s123.pth",
32
+ "model/model_s999.pth",
33
+ ],
34
+ )
35
+
36
+ img = Image.open("mammogram.png")
37
+ result = ens.predict(img) # averaged across 3 models
38
+ result = ens.predict_tta(img) # + test-time augmentation
39
+
40
+ print(result)
41
+ # { prediction, confidence, per_model, birads, modality }
42
+ """
43
+
44
+ from __future__ import annotations
45
+
46
+ import logging
47
+ from pathlib import Path
48
+ from typing import List, Optional, Union
49
+
50
+ import torch
51
+ import torch.nn as nn
52
+ from PIL import Image
53
+ from torchvision import models, transforms
54
+ from torchvision.models import EfficientNet_B4_Weights
55
+
56
+ logger = logging.getLogger(__name__)
57
+
58
+ MEAN = [0.485, 0.456, 0.406]
59
+ STD = [0.229, 0.224, 0.225]
60
+ TRAIN_SIZE = 384 # the models were trained at 384Γ—384
61
+
62
+
63
+ # ── Architecture (must exactly match the vast.ai training script) ──────────────
64
+ class _Model(nn.Module):
65
+ """EfficientNet-B4 single-view classifier β€” matches trained checkpoints."""
66
+
67
+ def __init__(self, pretrained: bool = False) -> None:
68
+ super().__init__()
69
+ w = EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None
70
+ bb = models.efficientnet_b4(weights=w)
71
+ self.feat = bb.features
72
+ self.pool = bb.avgpool
73
+ self.head = nn.Sequential(
74
+ nn.BatchNorm1d(1792), nn.Dropout(0.4),
75
+ nn.Linear(1792, 512), nn.ReLU(),
76
+ nn.BatchNorm1d(512), nn.Dropout(0.3),
77
+ nn.Linear(512, 2),
78
+ )
79
+
80
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
81
+ return self.head(torch.flatten(self.pool(self.feat(x)), 1))
82
+
83
+
84
+ def _build_val_transform(size: int = TRAIN_SIZE):
85
+ return transforms.Compose([
86
+ transforms.Resize((size, size)),
87
+ transforms.ToTensor(),
88
+ transforms.Normalize(MEAN, STD),
89
+ ])
90
+
91
+
92
+ def _build_tta_transforms(size: int = TRAIN_SIZE) -> List[transforms.Compose]:
93
+ base = [transforms.Resize((size, size)),
94
+ transforms.ToTensor(),
95
+ transforms.Normalize(MEAN, STD)]
96
+ return [
97
+ transforms.Compose(base),
98
+ transforms.Compose([transforms.RandomHorizontalFlip(p=1.0)] + base),
99
+ transforms.Compose([transforms.RandomVerticalFlip(p=1.0)] + base),
100
+ transforms.Compose([transforms.RandomRotation((10, 10))] + base),
101
+ ]
102
+
103
+
104
+ class MammogramEnsemble:
105
+ """
106
+ Averaging ensemble of the 3 trained EfficientNet-B4 mammogram models.
107
+
108
+ Parameters
109
+ ----------
110
+ weight_paths : list of paths to the 3 .pth checkpoints.
111
+ device : "cuda" | "mps" | "cpu". Auto-detected if None.
112
+ threshold : decision threshold for malignant. Default 0.5
113
+ (override with the ensemble threshold from results.json).
114
+ """
115
+
116
+ BIRADS_MAP = [
117
+ (0.90, "BI-RADS 5 β€” Highly suggestive of malignancy"),
118
+ (0.75, "BI-RADS 4C β€” High suspicion"),
119
+ (0.55, "BI-RADS 4B β€” Moderate suspicion"),
120
+ (0.40, "BI-RADS 4A β€” Low suspicion"),
121
+ ]
122
+
123
+ def __init__(
124
+ self,
125
+ weight_paths: List[Union[str, Path]],
126
+ device: Optional[str] = None,
127
+ threshold: float = 0.5,
128
+ ) -> None:
129
+ self.device = self._resolve_device(device)
130
+ self.threshold = threshold
131
+ self.transform = _build_val_transform()
132
+ self.tta_transforms = _build_tta_transforms()
133
+ self.models: List[_Model] = []
134
+ self.member_aucs: List[float] = []
135
+
136
+ for path in weight_paths:
137
+ path = Path(path)
138
+ if not path.exists():
139
+ logger.warning("Ensemble member not found: %s β€” skipping", path)
140
+ continue
141
+ m = _Model(pretrained=False).to(self.device)
142
+ ckpt = torch.load(path, map_location=self.device, weights_only=False)
143
+ state = ckpt.get("state_dict", ckpt)
144
+ m.load_state_dict(state, strict=True)
145
+ m.eval()
146
+ self.models.append(m)
147
+ auc = ckpt.get("metrics", {}).get("auc", 0.0)
148
+ self.member_aucs.append(float(auc) if isinstance(auc, (int, float)) else 0.0)
149
+ logger.info("Loaded %s (AUC=%s)", path.name, auc)
150
+
151
+ if not self.models:
152
+ raise RuntimeError("No ensemble members loaded β€” check weight paths.")
153
+
154
+ # Best-performing member, used for single-model tasks like Grad-CAM
155
+ # (saliency maps are inherently per-model; the strongest member is
156
+ # the most representative choice).
157
+ best_idx = int(max(range(len(self.models)),
158
+ key=lambda i: self.member_aucs[i]))
159
+ self.cam_model = self.models[best_idx]
160
+
161
+ logger.info("Ensemble ready: %d models on %s (CAM model: member %d)",
162
+ len(self.models), self.device, best_idx)
163
+
164
+ # ── Public API ────────────────────────────────────────────────────────────
165
+ def predict(self, image: Image.Image) -> dict:
166
+ """Average P(cancer) across all models for a single image."""
167
+ tensor = self.transform(image).unsqueeze(0).to(self.device)
168
+ per_model = []
169
+ with torch.inference_mode():
170
+ for m in self.models:
171
+ p = torch.softmax(m(tensor), dim=1)[0, 1].item()
172
+ per_model.append(round(p, 6))
173
+ return self._build_result(per_model)
174
+
175
+ def predict_tta(self, image: Image.Image) -> dict:
176
+ """Average across all models AND all TTA augmentations."""
177
+ per_model = []
178
+ with torch.inference_mode():
179
+ for m in self.models:
180
+ probs = []
181
+ for tf in self.tta_transforms:
182
+ t = tf(image).unsqueeze(0).to(self.device)
183
+ probs.append(torch.softmax(m(t), dim=1)[0, 1].item())
184
+ per_model.append(round(sum(probs) / len(probs), 6))
185
+ out = self._build_result(per_model)
186
+ out["modality"] = "mammogram_ensemble_tta"
187
+ return out
188
+
189
+ # ── Helpers ─────────────────────────────────────────────────────────────
190
+ def _build_result(self, per_model: List[float]) -> dict:
191
+ mal_conf = sum(per_model) / len(per_model)
192
+ prediction = "malignant" if mal_conf >= self.threshold else "benign"
193
+ confidence = mal_conf if prediction == "malignant" else (1 - mal_conf)
194
+ return {
195
+ "prediction": prediction,
196
+ "confidence": round(confidence, 6),
197
+ "malignant_probability": round(mal_conf, 6),
198
+ "per_model": per_model,
199
+ "birads": self._get_birads(prediction, mal_conf),
200
+ "modality": "mammogram_ensemble",
201
+ "n_models": len(per_model),
202
+ }
203
+
204
+ def _get_birads(self, prediction: str, mal_prob: float) -> str:
205
+ if prediction == "benign":
206
+ return ("BI-RADS 3 β€” Probably benign β€” short-interval follow-up"
207
+ if mal_prob >= 0.30 else "BI-RADS 2 β€” Benign finding")
208
+ for thr, label in self.BIRADS_MAP:
209
+ if mal_prob >= thr:
210
+ return label
211
+ return "BI-RADS 4A β€” Low suspicion"
212
+
213
+ @staticmethod
214
+ def _resolve_device(device: Optional[str]) -> torch.device:
215
+ if device is not None:
216
+ return torch.device(device)
217
+ if torch.cuda.is_available():
218
+ return torch.device("cuda")
219
+ if torch.backends.mps.is_available():
220
+ return torch.device("mps")
221
+ return torch.device("cpu")
222
+
223
+
224
+ # ── Quick self-test ────────────────────────────────────────────────────────────
225
+ if __name__ == "__main__":
226
+ logging.basicConfig(level=logging.INFO)
227
+ ens = MammogramEnsemble([
228
+ "model/model_s42.pth",
229
+ "model/model_s123.pth",
230
+ "model/model_s999.pth",
231
+ ])
232
+ # Random tensor sanity check β€” confirms all 3 load and run
233
+ dummy = Image.new("RGB", (512, 512), 128)
234
+ print(ens.predict(dummy))
model/mammogram_inference.py ADDED
@@ -0,0 +1,315 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ model/mammogram_inference.py
3
+ ─────────────────────────────
4
+ Inference pipeline for the multi-view EfficientNet-B4 mammogram classifier.
5
+
6
+ Innovation 4 β€” Test-Time Augmentation (TTA)
7
+ ─────────────────────────────────────────────
8
+ At inference time, each image is passed through the model 8 times with
9
+ different augmentations (flips and 90Β° rotations). The predicted
10
+ probabilities are averaged across all 8 runs.
11
+
12
+ Why TTA works:
13
+ The model never sees perfectly symmetrical versions of the same image
14
+ during training due to stochastic augmentation. At test time, presenting
15
+ multiple augmented versions and averaging "fills in" these unseen views,
16
+ effectively acting as a free ensemble of 8 models.
17
+
18
+ Expected improvement: +0.5–2% AUC with zero training cost.
19
+
20
+ Usage
21
+ ─────
22
+ from model.mammogram_inference import MammogramInferencePipeline
23
+ from PIL import Image
24
+
25
+ pipeline = MammogramInferencePipeline("model/mammogram_weights.pth")
26
+
27
+ # Standard prediction
28
+ img = Image.open("mammogram.png")
29
+ result = pipeline.predict(img)
30
+
31
+ # TTA prediction (slower but more accurate)
32
+ result = pipeline.predict_tta(img, n_augments=8)
33
+
34
+ # Multi-view prediction (CC + MLO)
35
+ cc_img = Image.open("cc.dcm")
36
+ mlo_img = Image.open("mlo.dcm")
37
+ result = pipeline.predict_multiview(cc_img, mlo_img)
38
+
39
+ print(result)
40
+ # { prediction, confidence, logits, birads, modality }
41
+ """
42
+
43
+ from __future__ import annotations
44
+
45
+ import logging
46
+ from pathlib import Path
47
+ from typing import List, Optional, Union
48
+
49
+ import torch
50
+ import torch.nn.functional as F
51
+ from PIL import Image
52
+ from torchvision import transforms
53
+
54
+ from model.mammogram_model import MultiViewMammogramClassifier
55
+ from utils.mammogram_preprocessing import (
56
+ build_mammogram_inference_transform,
57
+ load_mammogram,
58
+ )
59
+
60
+ logger = logging.getLogger(__name__)
61
+
62
+ # ── TTA augmentation set ──────────────────────────────────────────────────────
63
+ # 8 geometric transforms covering all flip/rotation combinations.
64
+ # No colour jitter β€” we want consistent intensity across augmentations.
65
+ def _build_tta_transforms(size: int = 512) -> List[transforms.Compose]:
66
+ """Return list of 8 TTA transforms (identity + 7 augmentations)."""
67
+ base = [
68
+ transforms.Resize((size, size)),
69
+ transforms.ToTensor(),
70
+ transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
71
+ ]
72
+ return [
73
+ transforms.Compose(base), # 1 original
74
+ transforms.Compose([transforms.RandomHorizontalFlip(p=1.0)] + base), # 2 hflip
75
+ transforms.Compose([transforms.RandomVerticalFlip(p=1.0)] + base), # 3 vflip
76
+ transforms.Compose([transforms.RandomRotation((90, 90))] + base), # 4 rot90
77
+ transforms.Compose([transforms.RandomRotation((180, 180))] + base), # 5 rot180
78
+ transforms.Compose([transforms.RandomRotation((270, 270))] + base), # 6 rot270
79
+ transforms.Compose([ # 7 hflip+rot90
80
+ transforms.RandomHorizontalFlip(p=1.0),
81
+ transforms.RandomRotation((90, 90)),
82
+ ] + base),
83
+ transforms.Compose([ # 8 vflip+rot90
84
+ transforms.RandomVerticalFlip(p=1.0),
85
+ transforms.RandomRotation((90, 90)),
86
+ ] + base),
87
+ ]
88
+
89
+
90
+ class MammogramInferencePipeline:
91
+ """
92
+ Multi-view EfficientNet-B4 inference pipeline with TTA support.
93
+
94
+ Parameters
95
+ ----------
96
+ weights_path : str | Path | None
97
+ Path to trained weights (.pth).
98
+ device : str | None
99
+ "cuda", "mps", or "cpu". Auto-detected if None.
100
+ confidence_threshold : float
101
+ Minimum probability to classify as malignant. Default 0.5.
102
+ use_tta : bool
103
+ Enable Test-Time Augmentation by default. Default False
104
+ (use predict_tta() explicitly for TTA).
105
+ """
106
+
107
+ BIRADS_MAP = [
108
+ (0.95, "BI-RADS 5 β€” Highly suggestive of malignancy"),
109
+ (0.85, "BI-RADS 4C β€” High suspicion"),
110
+ (0.75, "BI-RADS 4B β€” Moderate suspicion"),
111
+ (0.60, "BI-RADS 4A β€” Low suspicion"),
112
+ (0.50, "BI-RADS 3 β€” Probably benign β€” short follow-up"),
113
+ (0.00, "BI-RADS 2 β€” Benign finding"),
114
+ ]
115
+
116
+ def __init__(
117
+ self,
118
+ weights_path: Optional[Union[str, Path]] = None,
119
+ device: Optional[str] = None,
120
+ confidence_threshold: float = 0.5,
121
+ ) -> None:
122
+ self.device = self._resolve_device(device)
123
+ self.threshold = confidence_threshold
124
+ self.transform = build_mammogram_inference_transform()
125
+ self.tta_transforms = _build_tta_transforms()
126
+
127
+ # Build multi-view model
128
+ self.model = MultiViewMammogramClassifier(
129
+ pretrained = weights_path is None,
130
+ ).to(self.device)
131
+
132
+ if weights_path is not None:
133
+ weights_path = Path(weights_path)
134
+ if not weights_path.exists():
135
+ logger.warning(
136
+ "Mammogram weights not found at '%s'. "
137
+ "Using ImageNet init β€” predictions not clinically valid.",
138
+ weights_path,
139
+ )
140
+ else:
141
+ checkpoint = torch.load(
142
+ weights_path,
143
+ map_location = self.device,
144
+ weights_only = True,
145
+ )
146
+ state = checkpoint.get("state_dict", checkpoint)
147
+ self.model.load_state_dict(state, strict=True)
148
+ logger.info(
149
+ "Mammogram pipeline ready on %s (weights: %s)",
150
+ self.device, weights_path.name,
151
+ )
152
+ else:
153
+ logger.info(
154
+ "Mammogram pipeline ready on %s (ImageNet init)",
155
+ self.device,
156
+ )
157
+
158
+ self.model.eval()
159
+
160
+ # ── Public API ────────────────────────────────────────────────────────────
161
+
162
+ def predict(self, image: Image.Image) -> dict:
163
+ """
164
+ Standard single-image prediction (no TTA).
165
+
166
+ The image is used for both CC and MLO inputs β€” the model
167
+ makes the best prediction it can from a single view.
168
+ """
169
+ tensor = self.transform(image).unsqueeze(0).to(self.device)
170
+ with torch.inference_mode():
171
+ out = self.model.forward_single(tensor)
172
+ return self._build_result(out)
173
+
174
+ def predict_tta(
175
+ self,
176
+ image: Image.Image,
177
+ n_augments: int = 8,
178
+ ) -> dict:
179
+ """
180
+ Innovation 4 β€” Test-Time Augmentation prediction.
181
+
182
+ Runs the image through n_augments different augmentations,
183
+ averages the softmax probabilities, and returns the ensemble result.
184
+
185
+ Parameters
186
+ ----------
187
+ image : PIL image
188
+ n_augments : number of TTA augmentations (max 8). Default 8.
189
+
190
+ Returns
191
+ -------
192
+ Same dict schema as predict(), with modality="mammogram_tta".
193
+ """
194
+ n_augments = min(n_augments, len(self.tta_transforms))
195
+ all_probs = []
196
+
197
+ with torch.inference_mode():
198
+ for tf in self.tta_transforms[:n_augments]:
199
+ tensor = tf(image).unsqueeze(0).to(self.device)
200
+ out = self.model.forward_single(tensor)
201
+ all_probs.append(out["probs"])
202
+
203
+ # Average probabilities across all augmentations
204
+ avg_probs = torch.stack(all_probs).mean(0)
205
+ mal_conf = float(avg_probs[0, 1])
206
+ prediction = "malignant" if mal_conf >= self.threshold else "benign"
207
+ confidence = mal_conf if prediction == "malignant" else float(avg_probs[0, 0])
208
+
209
+ # Reconstruct result dict
210
+ fake_logits = torch.log(avg_probs + 1e-8)
211
+ return {
212
+ "prediction": prediction,
213
+ "confidence": round(confidence, 6),
214
+ "logits": fake_logits,
215
+ "birads": self._get_birads(prediction, mal_conf),
216
+ "modality": "mammogram_tta",
217
+ "n_augments": n_augments,
218
+ }
219
+
220
+ def predict_multiview(
221
+ self,
222
+ cc_image: Image.Image,
223
+ mlo_image: Image.Image,
224
+ ) -> dict:
225
+ """
226
+ Full multi-view prediction using CC and MLO views.
227
+
228
+ Parameters
229
+ ----------
230
+ cc_image : craniocaudal view PIL image
231
+ mlo_image : mediolateral oblique view PIL image
232
+ """
233
+ cc_tensor = self.transform(cc_image).unsqueeze(0).to(self.device)
234
+ mlo_tensor = self.transform(mlo_image).unsqueeze(0).to(self.device)
235
+
236
+ with torch.inference_mode():
237
+ out = self.model(cc_tensor, mlo_tensor)
238
+
239
+ result = self._build_result(out)
240
+ result["modality"] = "mammogram_multiview"
241
+ return result
242
+
243
+ def predict_multiview_tta(
244
+ self,
245
+ cc_image: Image.Image,
246
+ mlo_image: Image.Image,
247
+ n_augments: int = 8,
248
+ ) -> dict:
249
+ """Multi-view prediction with TTA β€” highest accuracy, slowest."""
250
+ n_augments = min(n_augments, len(self.tta_transforms))
251
+ all_probs = []
252
+
253
+ with torch.inference_mode():
254
+ for tf in self.tta_transforms[:n_augments]:
255
+ cc_t = tf(cc_image).unsqueeze(0).to(self.device)
256
+ mlo_t = tf(mlo_image).unsqueeze(0).to(self.device)
257
+ out = self.model(cc_t, mlo_t)
258
+ all_probs.append(out["probs"])
259
+
260
+ avg_probs = torch.stack(all_probs).mean(0)
261
+ mal_conf = float(avg_probs[0, 1])
262
+ prediction = "malignant" if mal_conf >= self.threshold else "benign"
263
+ confidence = mal_conf if prediction == "malignant" else float(avg_probs[0, 0])
264
+ fake_logits = torch.log(avg_probs + 1e-8)
265
+
266
+ return {
267
+ "prediction": prediction,
268
+ "confidence": round(confidence, 6),
269
+ "logits": fake_logits,
270
+ "birads": self._get_birads(prediction, mal_conf),
271
+ "modality": "mammogram_multiview_tta",
272
+ "n_augments": n_augments,
273
+ }
274
+
275
+ def predict_dicom(self, path: Union[str, Path]) -> dict:
276
+ """Load a DICOM file and run standard prediction."""
277
+ image = load_mammogram(path)
278
+ return self.predict(image)
279
+
280
+ def predict_dicom_tta(self, path: Union[str, Path]) -> dict:
281
+ """Load a DICOM file and run TTA prediction."""
282
+ image = load_mammogram(path)
283
+ return self.predict_tta(image)
284
+
285
+ def _get_birads(self, prediction: str, mal_prob: float) -> str:
286
+ if prediction == "benign":
287
+ return ("BI-RADS 3 β€” Probably benign β€” short-interval follow-up"
288
+ if mal_prob >= 0.35 else "BI-RADS 2 β€” Benign finding")
289
+ for threshold, label in self.BIRADS_MAP:
290
+ if mal_prob >= threshold:
291
+ return label
292
+ return "BI-RADS 2 β€” Benign finding"
293
+
294
+ def _build_result(self, out: dict) -> dict:
295
+ probs = out["probs"].squeeze()
296
+ mal_conf = float(probs[1])
297
+ prediction = "malignant" if mal_conf >= self.threshold else "benign"
298
+ confidence = mal_conf if prediction == "malignant" else float(probs[0])
299
+ return {
300
+ "prediction": prediction,
301
+ "confidence": round(confidence, 6),
302
+ "logits": out["logits"],
303
+ "birads": self._get_birads(prediction, mal_conf),
304
+ "modality": "mammogram",
305
+ }
306
+
307
+ @staticmethod
308
+ def _resolve_device(device: Optional[str]) -> torch.device:
309
+ if device is not None:
310
+ return torch.device(device)
311
+ if torch.cuda.is_available():
312
+ return torch.device("cuda")
313
+ if torch.backends.mps.is_available():
314
+ return torch.device("mps")
315
+ return torch.device("cpu")
model/mammogram_model.py ADDED
@@ -0,0 +1,293 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ model/mammogram_model.py
3
+ ─────────────────────────
4
+ EfficientNet-B4 mammogram classifier with five training innovations:
5
+
6
+ 1. Multi-View Patient Fusion
7
+ ── Siamese EfficientNet-B4 processes CC and MLO views with shared weights.
8
+ ── ViewAttentionFusion layer learns which view to trust more per case.
9
+ ── Patient-level prediction combines both breast views.
10
+ ── Clinical motivation: radiologists always read CC+MLO together.
11
+
12
+ 2. Focal Loss
13
+ ── Addresses severe class imbalance (1.5% cancer rate in RSNA).
14
+ ── FL(pt) = -Ξ±(1-pt)^Ξ³ Β· log(pt) β€” focuses on hard examples.
15
+ ── Prevents the model from collapsing to "predict everything benign".
16
+
17
+ 3. Mixed Precision Training (AMP)
18
+ ── torch.cuda.amp.autocast β€” 2Γ— faster, 40% less VRAM.
19
+ ── Enables larger batch sizes on A10G.
20
+ ── Used in training loop in modal_mammogram.py.
21
+
22
+ 4. Test-Time Augmentation (TTA)
23
+ ── Implemented in mammogram_inference.py.
24
+ ── 8 augmented predictions averaged at inference.
25
+
26
+ 5. Progressive Resizing
27
+ ── Implemented in modal_mammogram.py.
28
+ ── 256β†’384β†’512 curriculum during Phase 2 training.
29
+ """
30
+
31
+ from __future__ import annotations
32
+
33
+ import torch
34
+ import torch.nn as nn
35
+ import torch.nn.functional as F
36
+ from torchvision import models
37
+ from torchvision.models import EfficientNet_B4_Weights
38
+
39
+
40
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
41
+ # β•‘ INNOVATION 1 β€” VIEW ATTENTION FUSION β•‘
42
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
43
+
44
+ class ViewAttentionFusion(nn.Module):
45
+ """
46
+ Soft-attention fusion of CC and MLO mammogram view features.
47
+
48
+ Rather than naively averaging CC and MLO predictions, this module
49
+ learns a per-sample attention weight β€” how much to trust each view.
50
+
51
+ In benign cases the CC view often dominates.
52
+ In malignant cases the view with the visible lesion drives the prediction.
53
+ The attention mechanism learns this distinction from data.
54
+
55
+ Architecture
56
+ ────────────
57
+ Input: cc_feat (B, D) β€” EfficientNet features from CC view
58
+ mlo_feat (B, D) β€” EfficientNet features from MLO view
59
+
60
+ Gate: [cc_feat | mlo_feat] β†’ Linear(2D, 2) β†’ Softmax β†’ (Ξ±_cc, Ξ±_mlo)
61
+
62
+ Fusion: fused = Ξ±_cc Β· cc_feat + Ξ±_mlo Β· mlo_feat
63
+
64
+ Residual gating adds a skip connection so the fusion can fall back
65
+ to simple averaging if attention provides no benefit.
66
+
67
+ Parameters
68
+ ----------
69
+ dim : int
70
+ Feature dimension (1792 for EfficientNet-B4).
71
+ dropout : float
72
+ Dropout on the gate network. Default 0.2.
73
+ """
74
+
75
+ def __init__(self, dim: int = 1792, dropout: float = 0.2) -> None:
76
+ super().__init__()
77
+ self.gate = nn.Sequential(
78
+ nn.Linear(dim * 2, dim // 4),
79
+ nn.ReLU(inplace=True),
80
+ nn.Dropout(dropout),
81
+ nn.Linear(dim // 4, 2),
82
+ )
83
+ # Learnable residual blend β€” starts at 0.5/0.5
84
+ self.residual_w = nn.Parameter(torch.tensor(0.5))
85
+
86
+ def forward(
87
+ self,
88
+ cc_feat: torch.Tensor, # (B, D)
89
+ mlo_feat: torch.Tensor, # (B, D)
90
+ ) -> torch.Tensor: # (B, D)
91
+ concat = torch.cat([cc_feat, mlo_feat], dim=-1) # (B, 2D)
92
+ weights = torch.softmax(self.gate(concat), dim=-1) # (B, 2)
93
+ Ξ±_cc = weights[:, 0:1] # (B, 1)
94
+ Ξ±_mlo = weights[:, 1:2] # (B, 1)
95
+
96
+ # Attention-weighted fusion + residual simple-average
97
+ attended = Ξ±_cc * cc_feat + Ξ±_mlo * mlo_feat
98
+ simple = 0.5 * cc_feat + 0.5 * mlo_feat
99
+ w = torch.sigmoid(self.residual_w)
100
+ return w * attended + (1 - w) * simple
101
+
102
+
103
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
104
+ # β•‘ INNOVATION 2 β€” FOCAL LOSS β•‘
105
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
106
+
107
+ class FocalLoss(nn.Module):
108
+ """
109
+ Focal Loss for extreme class imbalance (Lin et al., 2017).
110
+
111
+ RSNA 2022 has ~1.5% cancer rate. Standard cross-entropy loss with
112
+ pos_weight is a blunt instrument β€” it upweights all positive samples
113
+ equally. Focal Loss instead downweights easy negative examples,
114
+ forcing the model to focus on hard-to-classify borderline cases.
115
+
116
+ FL(pt) = -Ξ± Β· (1 - pt)^Ξ³ Β· log(pt)
117
+
118
+ Parameters
119
+ ----------
120
+ alpha : float
121
+ Balance parameter. 0.25 focuses on positives. Default 0.25.
122
+ gamma : float
123
+ Focusing parameter. 0 = standard CE, 2 = standard focal. Default 2.
124
+ pos_weight : torch.Tensor | None
125
+ Class weight for the positive (cancer) class. Multiplies alpha.
126
+ label_smoothing : float
127
+ Prevents overconfident predictions. Default 0.1.
128
+ """
129
+
130
+ def __init__(
131
+ self,
132
+ alpha: float = 0.25,
133
+ gamma: float = 2.0,
134
+ pos_weight: torch.Tensor | None = None,
135
+ label_smoothing: float = 0.1,
136
+ ) -> None:
137
+ super().__init__()
138
+ self.alpha = alpha
139
+ self.gamma = gamma
140
+ self.pos_weight = pos_weight
141
+ self.label_smoothing = label_smoothing
142
+
143
+ def forward(
144
+ self,
145
+ logits: torch.Tensor, # (B, C)
146
+ targets: torch.Tensor, # (B,) long
147
+ ) -> torch.Tensor:
148
+ # Standard CE with label smoothing and class weights
149
+ ce = F.cross_entropy(
150
+ logits, targets,
151
+ weight = self.pos_weight,
152
+ label_smoothing = self.label_smoothing,
153
+ reduction = "none",
154
+ )
155
+ # Focal modulation β€” downweight easy examples
156
+ pt = torch.exp(-ce)
157
+ focal_loss = self.alpha * (1 - pt) ** self.gamma * ce
158
+ return focal_loss.mean()
159
+
160
+
161
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
162
+ # β•‘ MULTI-VIEW MAMMOGRAM CLASSIFIER β•‘
163
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
164
+
165
+ class MultiViewMammogramClassifier(nn.Module):
166
+ """
167
+ Siamese EfficientNet-B4 with attention-based CC+MLO view fusion.
168
+
169
+ Both views are processed by the same EfficientNet-B4 backbone
170
+ (weight sharing). The ViewAttentionFusion layer then combines
171
+ the two feature vectors with a learned attention gate.
172
+
173
+ Parameters
174
+ ----------
175
+ pretrained : bool
176
+ Load ImageNet weights. Default True.
177
+ freeze_backbone : bool
178
+ Freeze backbone during Phase 1 training. Default False.
179
+ dropout_rate : float
180
+ Dropout in the classifier head. Default 0.4.
181
+ """
182
+
183
+ FEATURE_DIM = 1792 # EfficientNet-B4 output channels
184
+
185
+ def __init__(
186
+ self,
187
+ pretrained: bool = True,
188
+ freeze_backbone: bool = False,
189
+ dropout_rate: float = 0.4,
190
+ ) -> None:
191
+ super().__init__()
192
+
193
+ # Shared backbone β€” same weights for both views (Siamese)
194
+ weights = EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None
195
+ backbone = models.efficientnet_b4(weights=weights)
196
+ self.features = backbone.features
197
+ self.avgpool = backbone.avgpool
198
+
199
+ # Freeze for Phase 1
200
+ if freeze_backbone:
201
+ for p in self.features.parameters():
202
+ p.requires_grad = False
203
+
204
+ # View attention fusion
205
+ self.fusion = ViewAttentionFusion(dim=self.FEATURE_DIM)
206
+
207
+ # Classifier head
208
+ D = self.FEATURE_DIM
209
+ self.classifier = nn.Sequential(
210
+ nn.BatchNorm1d(D),
211
+ nn.Dropout(p=dropout_rate),
212
+ nn.Linear(D, 512),
213
+ nn.ReLU(inplace=True),
214
+ nn.BatchNorm1d(512),
215
+ nn.Dropout(p=dropout_rate * 0.75),
216
+ nn.Linear(512, 2),
217
+ )
218
+
219
+ def encode(self, x: torch.Tensor) -> torch.Tensor:
220
+ """Encode a single view to a feature vector (B, D)."""
221
+ feat = self.features(x) # (B, D, H, W)
222
+ pooled = self.avgpool(feat) # (B, D, 1, 1)
223
+ return torch.flatten(pooled, 1) # (B, D)
224
+
225
+ def forward(
226
+ self,
227
+ cc_view: torch.Tensor, # (B, 3, H, W) β€” craniocaudal view
228
+ mlo_view: torch.Tensor, # (B, 3, H, W) β€” mediolateral oblique view
229
+ ) -> dict[str, torch.Tensor]:
230
+ """
231
+ Forward pass with both views.
232
+
233
+ Returns dict with "logits" (B,2) and "probs" (B,2).
234
+ """
235
+ cc_feat = self.encode(cc_view) # (B, D)
236
+ mlo_feat = self.encode(mlo_view) # (B, D)
237
+ fused = self.fusion(cc_feat, mlo_feat)# (B, D)
238
+ logits = self.classifier(fused) # (B, 2)
239
+ probs = torch.softmax(logits, dim=1)
240
+ return {"logits": logits, "probs": probs}
241
+
242
+ def forward_single(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
243
+ """
244
+ Single-view forward pass β€” duplicates input for both CC and MLO.
245
+ Used at inference when only one image is available.
246
+ """
247
+ return self.forward(x, x)
248
+
249
+
250
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
251
+ # β•‘ SINGLE-VIEW CLASSIFIER (kept for backward compatibility) β•‘
252
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
253
+
254
+ class MammogramClassifier(nn.Module):
255
+ """
256
+ Single-view EfficientNet-B4 (original architecture).
257
+ Kept for backward compatibility with existing inference code.
258
+ """
259
+
260
+ def __init__(
261
+ self,
262
+ pretrained: bool = True,
263
+ freeze_backbone: bool = False,
264
+ dropout_rate: float = 0.4,
265
+ num_classes: int = 2,
266
+ ) -> None:
267
+ super().__init__()
268
+ weights = EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None
269
+ backbone = models.efficientnet_b4(weights=weights)
270
+ self.features = backbone.features
271
+ self.avgpool = backbone.avgpool
272
+
273
+ if freeze_backbone:
274
+ for p in self.features.parameters():
275
+ p.requires_grad = False
276
+
277
+ D = 1792
278
+ self.classifier = nn.Sequential(
279
+ nn.BatchNorm1d(D),
280
+ nn.Dropout(p=dropout_rate),
281
+ nn.Linear(D, 512),
282
+ nn.ReLU(inplace=True),
283
+ nn.BatchNorm1d(512),
284
+ nn.Dropout(p=dropout_rate * 0.75),
285
+ nn.Linear(512, num_classes),
286
+ )
287
+
288
+ def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
289
+ feat = self.features(x)
290
+ pooled = self.avgpool(feat)
291
+ flat = torch.flatten(pooled, 1)
292
+ logits = self.classifier(flat)
293
+ return {"logits": logits, "probs": torch.softmax(logits, dim=1)}
model/model.py ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ model/model.py
3
+ ──────────────
4
+ DenseNet-121 backbone fine-tuned for binary classification of
5
+ breast cancer histopathology images (benign vs. malignant).
6
+
7
+ Architecture
8
+ ────────────
9
+ DenseNet-121 (pretrained on ImageNet)
10
+ └─ Adaptive Average Pool β†’ flatten
11
+ └─ Classifier head
12
+ β”œβ”€ BatchNorm1d(1024)
13
+ β”œβ”€ Dropout(p=0.4)
14
+ β”œβ”€ Linear(1024 β†’ 256) + ReLU
15
+ β”œβ”€ BatchNorm1d(256)
16
+ β”œβ”€ Dropout(p=0.3)
17
+ └─ Linear(256 β†’ 2) ← raw logits [benign, malignant]
18
+
19
+ Outputs
20
+ ────────
21
+ logits : Tensor[1, 2] β€” raw scores (pre-softmax)
22
+ probs : Tensor[1, 2] β€” calibrated probabilities via softmax
23
+ """
24
+
25
+ import torch
26
+ import torch.nn as nn
27
+ from torchvision import models
28
+
29
+
30
+ class BreastCancerClassifier(nn.Module):
31
+ """
32
+ DenseNet-121 backbone with a custom two-class head.
33
+
34
+ Parameters
35
+ ----------
36
+ pretrained : bool
37
+ Load ImageNet weights into the DenseNet-121 backbone (default True).
38
+ freeze_backbone : bool
39
+ Freeze all DenseNet layers except the classifier head (default False).
40
+ Set True for pure feature-extraction / fast fine-tuning scenarios.
41
+ dropout_rate : float
42
+ Dropout probability applied in the classifier head (default 0.4).
43
+ """
44
+
45
+ def __init__(
46
+ self,
47
+ pretrained: bool = True,
48
+ freeze_backbone: bool = False,
49
+ dropout_rate: float = 0.4,
50
+ ) -> None:
51
+ super().__init__()
52
+
53
+ # ── Backbone ────────────────────────────────────────────────────────
54
+ weights = models.DenseNet121_Weights.IMAGENET1K_V1 if pretrained else None
55
+ densenet = models.densenet121(weights=weights)
56
+
57
+ # Keep every layer except the original FC classifier
58
+ self.features = densenet.features # Conv + DenseBlocks + Transitions
59
+ self.pool = nn.AdaptiveAvgPool2d((1, 1))
60
+
61
+ in_features = densenet.classifier.in_features # 1024 for DenseNet-121
62
+
63
+ # ── Classifier head ─────────────────────────────────────────────────
64
+ self.classifier = nn.Sequential(
65
+ nn.BatchNorm1d(in_features),
66
+ nn.Dropout(p=dropout_rate),
67
+ nn.Linear(in_features, 256),
68
+ nn.ReLU(inplace=True),
69
+ nn.BatchNorm1d(256),
70
+ nn.Dropout(p=dropout_rate * 0.75),
71
+ nn.Linear(256, 2), # 2 logits: [benign, malignant]
72
+ )
73
+
74
+ # ── Optional backbone freeze ─────────────────────────────────────────
75
+ if freeze_backbone:
76
+ for param in self.features.parameters():
77
+ param.requires_grad = False
78
+
79
+ self._init_classifier_weights()
80
+
81
+ # ────────────────────────────────────────────────────────────────────────
82
+ def _init_classifier_weights(self) -> None:
83
+ """Kaiming / Xavier initialisation for the custom head."""
84
+ for module in self.classifier.modules():
85
+ if isinstance(module, nn.Linear):
86
+ nn.init.kaiming_normal_(module.weight, nonlinearity="relu")
87
+ if module.bias is not None:
88
+ nn.init.zeros_(module.bias)
89
+ elif isinstance(module, nn.BatchNorm1d):
90
+ nn.init.ones_(module.weight)
91
+ nn.init.zeros_(module.bias)
92
+
93
+ # ────────────────────────────────────────────────────────────────────────
94
+ def forward(self, x: torch.Tensor) -> dict:
95
+ """
96
+ Forward pass.
97
+
98
+ Parameters
99
+ ----------
100
+ x : torch.Tensor
101
+ Normalised image tensor of shape (B, 3, 224, 224).
102
+
103
+ Returns
104
+ -------
105
+ dict with keys
106
+ "logits" : Tensor[B, 2] β€” raw model outputs
107
+ "probs" : Tensor[B, 2] β€” softmax probabilities
108
+ """
109
+ features = self.features(x) # (B, 1024, 7, 7)
110
+ pooled = self.pool(features) # (B, 1024, 1, 1)
111
+ flat = torch.flatten(pooled, 1) # (B, 1024)
112
+ logits = self.classifier(flat) # (B, 2)
113
+ probs = torch.softmax(logits, dim=1) # (B, 2)
114
+
115
+ return {"logits": logits, "probs": probs}
116
+
117
+ # ────────────────────────────────────────────────────────────────────────
118
+ def get_feature_maps(self, x: torch.Tensor) -> torch.Tensor:
119
+ """
120
+ Return the final DenseNet feature maps before pooling.
121
+ Used by Grad-CAM and other spatial explainability modules.
122
+
123
+ Returns
124
+ -------
125
+ Tensor[B, 1024, 7, 7]
126
+ """
127
+ return self.features(x)
model/model_s123.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99bc8f2c7f49b95c2645d7f714ad21bb66543c2df15b1d06b33433e012259c6c
3
+ size 74664479
model/model_s42.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75c651eb9f644556be8f28da02f36c4e5bc3c9774abad38513e46bb6aecf5540
3
+ size 74663755
model/model_s999.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8336c6919dd7dfe9133544685568993250db58fea3cc5afa4909fe57c93c9e61
3
+ size 74664479
model/weights.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9cbb6abcfcd8931e2f62293def9fbce49beb709430565603ac7d97d53918a38
3
+ size 87564433
requirements-deploy.txt ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MedAI backend β€” SERVING dependencies only (Hugging Face Space).
2
+ # Torch CPU wheels are installed separately in the Dockerfile (smaller than CUDA).
3
+ # Excludes training/FLAN-T5 deps (transformers, datasets, accelerate, sentencepiece)
4
+ # because the API defaults to the deterministic template explainer (USE_LLM_MODEL=false).
5
+
6
+ Pillow>=10.0.0
7
+ numpy>=1.24.0,<2.0
8
+ scipy>=1.11.0
9
+
10
+ # Mammogram DICOM support
11
+ pydicom>=2.4.0
12
+ pylibjpeg>=2.0.0
13
+ python-gdcm>=3.0.0
14
+ opencv-python-headless>=4.8.0
15
+
16
+ # API
17
+ fastapi>=0.110.0
18
+ uvicorn[standard]>=0.29.0
19
+ python-multipart>=0.0.9
20
+ python-dotenv>=1.0.0
21
+
22
+ # Chat (Groq via the OpenAI-compatible SDK)
23
+ openai>=1.30.0
24
+ requests>=2.31.0
requirements.txt ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Core ML
2
+ torch>=2.1.0
3
+ torchvision>=0.16.0
4
+ Pillow>=10.0.0
5
+ numpy>=1.24.0,<2.0
6
+
7
+ # Histopathology dataset
8
+ datasets>=2.18.0
9
+
10
+ # LLM Explainer
11
+ transformers>=4.38.0
12
+ sentencepiece>=0.1.99
13
+ accelerate>=0.27.0
14
+
15
+ # Mammogram DICOM support
16
+ pydicom>=2.4.0
17
+ pylibjpeg>=2.0.0
18
+ python-gdcm>=3.0.0
19
+ scipy>=1.11.0
20
+ opencv-python-headless>=4.8.0
21
+
22
+ # API
23
+ fastapi>=0.110.0
24
+ uvicorn[standard]>=0.29.0
25
+ python-multipart>=0.0.9
26
+ python-dotenv>=1.0.0
27
+
28
+ # Chat pipeline
29
+ groq>=0.9.0
30
+ requests>=2.31.0
31
+
32
+ # Testing
33
+ pytest>=8.0.0
tests/test_classifier.py ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ tests/test_classifier.py
3
+ ────────────────────────
4
+ Unit and integration tests for the medical-ai module.
5
+
6
+ Run from the repo root:
7
+ pytest tests/ -v
8
+
9
+ Import paths assume the project root (medical-ai/) is the working directory.
10
+ """
11
+
12
+ import sys
13
+ import os
14
+ sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
15
+
16
+ import numpy as np
17
+ import pytest
18
+ import torch
19
+ from PIL import Image
20
+
21
+ from model import BreastCancerClassifier, BreastCancerInferencePipeline
22
+ from utils import ImagePreprocessor
23
+
24
+
25
+ # ── Fixtures ──────────────────────────────────────────────────────────────────
26
+
27
+ @pytest.fixture(scope="module")
28
+ def dummy_pil():
29
+ """224Γ—224 RGB PIL image filled with mid-grey."""
30
+ return Image.fromarray(
31
+ np.full((224, 224, 3), 128, dtype=np.uint8), mode="RGB"
32
+ )
33
+
34
+ @pytest.fixture(scope="module")
35
+ def dummy_np():
36
+ """224Γ—224Γ—3 uint8 numpy array."""
37
+ return np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
38
+
39
+ @pytest.fixture(scope="module")
40
+ def dummy_tensor():
41
+ """Properly-shaped (1, 3, 224, 224) float32 tensor."""
42
+ return torch.rand(1, 3, 224, 224)
43
+
44
+ @pytest.fixture(scope="module")
45
+ def model():
46
+ return BreastCancerClassifier(pretrained=False)
47
+
48
+ @pytest.fixture(scope="module")
49
+ def pipeline():
50
+ return BreastCancerInferencePipeline(weights_path=None, device="cpu")
51
+
52
+
53
+ # ── model/model.py ────────────────────────────────────────────────────────────
54
+
55
+ class TestBreastCancerClassifier:
56
+
57
+ def test_output_keys(self, model):
58
+ x = torch.rand(1, 3, 224, 224)
59
+ out = model(x)
60
+ assert "logits" in out and "probs" in out
61
+
62
+ def test_logits_shape(self, model):
63
+ x = torch.rand(2, 3, 224, 224)
64
+ out = model(x)
65
+ assert out["logits"].shape == (2, 2), "Logits must be (B, 2)"
66
+
67
+ def test_probs_sum_to_one(self, model):
68
+ x = torch.rand(4, 3, 224, 224)
69
+ out = model(x)
70
+ sums = out["probs"].sum(dim=1)
71
+ assert torch.allclose(sums, torch.ones(4), atol=1e-5)
72
+
73
+ def test_probs_in_range(self, model):
74
+ x = torch.rand(1, 3, 224, 224)
75
+ out = model(x)
76
+ assert out["probs"].min() >= 0.0
77
+ assert out["probs"].max() <= 1.0
78
+
79
+ def test_feature_maps_shape(self, model):
80
+ """Validates Grad-CAM hook compatibility."""
81
+ x = torch.rand(1, 3, 224, 224)
82
+ fm = model.get_feature_maps(x)
83
+ assert fm.shape == (1, 1024, 7, 7)
84
+
85
+ def test_deterministic_output(self, model):
86
+ """Identical inputs must produce identical outputs at eval time."""
87
+ model.eval()
88
+ x = torch.rand(1, 3, 224, 224)
89
+ with torch.no_grad():
90
+ out_a = model(x)
91
+ out_b = model(x)
92
+ assert torch.allclose(out_a["logits"], out_b["logits"])
93
+
94
+ def test_batch_inference(self, model):
95
+ x = torch.rand(8, 3, 224, 224)
96
+ out = model(x)
97
+ assert out["logits"].shape == (8, 2)
98
+
99
+ def test_freeze_backbone(self):
100
+ m = BreastCancerClassifier(pretrained=False, freeze_backbone=True)
101
+ for param in m.features.parameters():
102
+ assert not param.requires_grad, "Backbone params should be frozen"
103
+
104
+
105
+ # ── utils/preprocessing.py ───────────────────────────────────────────────────
106
+
107
+ class TestImagePreprocessor:
108
+
109
+ def test_pil_input(self, dummy_pil):
110
+ t = ImagePreprocessor()(dummy_pil)
111
+ assert t.shape == (1, 3, 224, 224)
112
+ assert t.dtype == torch.float32
113
+
114
+ def test_numpy_input(self, dummy_np):
115
+ t = ImagePreprocessor()(dummy_np)
116
+ assert t.shape == (1, 3, 224, 224)
117
+
118
+ def test_tensor_input(self, dummy_tensor):
119
+ t = ImagePreprocessor()(dummy_tensor)
120
+ assert t.shape == (1, 3, 224, 224)
121
+
122
+ def test_normalization_shifts_range(self, dummy_pil):
123
+ """ImageNet normalization should shift values outside raw [0,1]."""
124
+ t = ImagePreprocessor()(dummy_pil)
125
+ assert not (t.min() >= 0 and t.max() <= 1)
126
+
127
+ def test_invalid_type_raises(self):
128
+ with pytest.raises(TypeError):
129
+ ImagePreprocessor()({"not": "an image"})
130
+
131
+ def test_grayscale_numpy_converted_to_rgb(self):
132
+ grey = np.full((224, 224), 128, dtype=np.uint8)
133
+ t = ImagePreprocessor()(grey)
134
+ assert t.shape == (1, 3, 224, 224)
135
+
136
+
137
+ # ── model/inference.py ────────────────────────────────────────────────────────
138
+
139
+ class TestInferencePipeline:
140
+
141
+ def test_output_schema(self, pipeline, dummy_pil):
142
+ result = pipeline.predict(dummy_pil)
143
+ assert "prediction" in result
144
+ assert "confidence" in result
145
+ assert "logits" in result
146
+
147
+ def test_prediction_is_valid_label(self, pipeline, dummy_pil):
148
+ result = pipeline.predict(dummy_pil)
149
+ assert result["prediction"] in ("benign", "malignant")
150
+
151
+ def test_confidence_range(self, pipeline, dummy_pil):
152
+ result = pipeline.predict(dummy_pil)
153
+ assert 0.0 <= result["confidence"] <= 1.0
154
+
155
+ def test_logits_shape(self, pipeline, dummy_pil):
156
+ result = pipeline.predict(dummy_pil)
157
+ assert result["logits"].shape == (1, 2)
158
+
159
+ def test_batch_predict_length(self, pipeline, dummy_pil, dummy_np):
160
+ results = pipeline.predict_batch([dummy_pil, dummy_np])
161
+ assert len(results) == 2
162
+
163
+ def test_deterministic_inference(self, pipeline, dummy_pil):
164
+ r1 = pipeline.predict(dummy_pil)
165
+ r2 = pipeline.predict(dummy_pil)
166
+ assert r1["prediction"] == r2["prediction"]
167
+ assert r1["confidence"] == r2["confidence"]
168
+ assert torch.allclose(r1["logits"], r2["logits"])
169
+
170
+ def test_numpy_input_accepted(self, pipeline, dummy_np):
171
+ result = pipeline.predict(dummy_np)
172
+ assert result["prediction"] in ("benign", "malignant")
173
+
174
+ def test_logits_detached_from_graph(self, pipeline, dummy_pil):
175
+ result = pipeline.predict(dummy_pil)
176
+ assert not result["logits"].requires_grad
177
+
178
+ def test_missing_weights_raises(self):
179
+ with pytest.raises(FileNotFoundError):
180
+ BreastCancerInferencePipeline(
181
+ weights_path="model/nonexistent_weights.pth",
182
+ device="cpu",
183
+ )
train.py ADDED
@@ -0,0 +1,692 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ train.py
3
+ ────────
4
+ Training script for the BreastCancerClassifier (DenseNet-121).
5
+
6
+ What this file does
7
+ ───────────────────
8
+ 1. Loads the BreastMNIST dataset (auto-downloaded via MedMNIST)
9
+ 2. Applies training augmentations from utils/preprocessing.py
10
+ 3. Fine-tunes the DenseNet-121 backbone + custom classifier head
11
+ 4. Validates after every epoch and tracks the best model
12
+ 5. Saves the best weights to model/weights.pth
13
+
14
+ Usage
15
+ ─────
16
+ # Basic β€” runs with all defaults
17
+ python train.py
18
+
19
+ # Custom β€” override any hyperparameter
20
+ python train.py --epochs 30 --lr 1e-4 --batch-size 64 --freeze-backbone
21
+
22
+ Output
23
+ ──────
24
+ model/weights.pth ← best checkpoint (loaded by inference.py)
25
+ training_log.csv ← per-epoch metrics for plotting
26
+
27
+ Dataset
28
+ ───────
29
+ PatchCamelyon / PCam (HuggingFace β€” 1aurent/PatchCamelyon)
30
+ - 327,680 training patches (roughly balanced ~50/50)
31
+ - 32,768 validation patches
32
+ - 32,768 test patches
33
+ - Binary labels: 0 = no tumour tissue, 1 = tumour tissue present
34
+ - 96Γ—96 px RGB H&E-stained histopathology patches
35
+ - Sourced from Camelyon16 whole-slide images (lymph node sections)
36
+
37
+ Install dependencies
38
+ ────────────────────
39
+ pip install -r requirements.txt
40
+ pip install datasets Pillow
41
+ """
42
+
43
+ from __future__ import annotations
44
+
45
+ import argparse
46
+ import csv
47
+ import logging
48
+ import sys
49
+ import time
50
+ from pathlib import Path
51
+
52
+ import torch
53
+ import torch.nn as nn
54
+ from torch.optim import Adam
55
+ from torch.optim.lr_scheduler import OneCycleLR
56
+ from torch.utils.data import DataLoader
57
+
58
+ # ── Project imports ──────────────────────────────────────────────────────────
59
+ # Insert project root so `model` and `utils` are importable regardless of
60
+ # where you invoke the script from.
61
+ ROOT = Path(__file__).resolve().parent
62
+ sys.path.insert(0, str(ROOT))
63
+
64
+ from model.model import BreastCancerClassifier
65
+ from utils.preprocessing import build_training_transform, build_inference_transform
66
+
67
+ # ── Logging ───────────────────────────────────────────────────────────────────
68
+ logging.basicConfig(
69
+ level=logging.INFO,
70
+ format="%(asctime)s %(levelname)-8s %(message)s",
71
+ datefmt="%H:%M:%S",
72
+ )
73
+ logger = logging.getLogger(__name__)
74
+
75
+
76
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
77
+ # β•‘ CONFIGURATION β•‘
78
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
79
+
80
+ def parse_args() -> argparse.Namespace:
81
+ """All hyperparameters are CLI-overridable β€” nothing is hardcoded."""
82
+ p = argparse.ArgumentParser(
83
+ description="Train BreastCancerClassifier on BreastMNIST",
84
+ formatter_class=argparse.ArgumentDefaultsHelpFormatter,
85
+ )
86
+
87
+ # ── Data ─────────────────────────────────────────────────────────────────
88
+ p.add_argument("--data-dir", type=str, default="data",
89
+ help="Directory to download / cache PatchCamelyon")
90
+
91
+ # ── Training ─────────────────────────────────────────────────────────────
92
+ p.add_argument("--epochs", type=int, default=20,
93
+ help="Total training epochs")
94
+ p.add_argument("--batch-size", type=int, default=32,
95
+ help="Samples per mini-batch")
96
+ p.add_argument("--lr", type=float, default=3e-4,
97
+ help="Initial learning rate for Adam optimizer")
98
+ p.add_argument("--weight-decay", type=float, default=1e-4,
99
+ help="L2 regularization strength")
100
+
101
+ # ── Model ────────────────────────────────────────────────────────────────
102
+ p.add_argument("--freeze-backbone", action="store_true",
103
+ help="Freeze DenseNet feature layers β€” train head only")
104
+ p.add_argument("--dropout", type=float, default=0.4,
105
+ help="Dropout rate in classifier head")
106
+
107
+ # ── LR Scheduler ─────────���───────────────────────────────────────────────
108
+ p.add_argument("--lr-patience", type=int, default=4,
109
+ help="Epochs with no val-loss improvement before LR reduction")
110
+ p.add_argument("--lr-factor", type=float, default=0.5,
111
+ help="Multiply LR by this factor on plateau")
112
+
113
+ # ── Output ───────────────────────────────────────────────────────────────
114
+ p.add_argument("--weights-out", type=str, default="model/weights.pth",
115
+ help="Path to save the best model checkpoint")
116
+ p.add_argument("--log-out", type=str, default="training_log.csv",
117
+ help="Path to save per-epoch metrics CSV")
118
+
119
+ # ── Misc ─────────────────────────────────────────────────────────────────
120
+ p.add_argument("--num-workers", type=int, default=4,
121
+ help="DataLoader worker threads")
122
+ p.add_argument("--seed", type=int, default=42,
123
+ help="Random seed for reproducibility")
124
+ p.add_argument("--early-stop", type=int, default=7,
125
+ help="Stop if val loss doesn't improve for N epochs (0 = off)")
126
+
127
+ # ── Regularization fixes ──────────────────────────────────────────────────
128
+ p.add_argument("--mixup-alpha", type=float, default=0.4,
129
+ help="Beta distribution alpha for Mixup (0 = disabled)")
130
+ p.add_argument("--label-smoothing", type=float, default=0.1,
131
+ help="Label smoothing for CrossEntropyLoss (0 = disabled)")
132
+
133
+ # ── OneCycleLR ────────────────────────────────────────────────────────────
134
+ p.add_argument("--max-lr", type=float, default=3e-3,
135
+ help="Peak LR for OneCycleLR (typically 10x base lr)")
136
+
137
+ # ── Deduplication ─────────────────────────────────────────────────────────
138
+ p.add_argument("--deduplicate", action="store_true",
139
+ help="Remove duplicate images from training set via MD5 hashing")
140
+
141
+ return p.parse_args()
142
+
143
+
144
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
145
+ # β•‘ DATASET β•‘
146
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
147
+
148
+ def get_dataloaders(
149
+ args: argparse.Namespace,
150
+ ) -> tuple[DataLoader, DataLoader, DataLoader, torch.Tensor]:
151
+ """
152
+ Stream PatchCamelyon from HuggingFace and return
153
+ (train_loader, val_loader, test_loader, pos_weight).
154
+
155
+ PatchCamelyon (PCam)
156
+ ────────────────────
157
+ 327,680 train / 32,768 val / 32,768 test
158
+ 96Γ—96 px RGB H&E-stained histopathology patches from lymph node sections.
159
+ Label 0 = no tumour tissue in centre region.
160
+ Label 1 = tumour tissue present in centre region.
161
+ Dataset is roughly balanced (~50/50) so pos_weight will be close to 1.0.
162
+
163
+ HuggingFace streaming
164
+ ─────────────────────
165
+ PCam is ~7 GB total. cache_dir stores the downloaded files locally so
166
+ subsequent runs don't re-download. On first run this may take a few minutes
167
+ depending on your connection speed.
168
+ """
169
+ try:
170
+ from datasets import load_dataset
171
+ except ImportError:
172
+ logger.error("datasets not installed. Run: pip install datasets")
173
+ sys.exit(1)
174
+
175
+ data_dir = Path(args.data_dir)
176
+ data_dir.mkdir(parents=True, exist_ok=True)
177
+
178
+ logger.info("Loading PatchCamelyon from HuggingFace (cache: %s) ...", data_dir)
179
+ logger.info("First run will download ~7 GB β€” subsequent runs use cache.")
180
+
181
+ # ── Download / load all three splits ─────────────────────────────────────
182
+ # cache_dir stores the Arrow files locally after first download.
183
+ hf_train = load_dataset("1aurent/PatchCamelyon", split="train",
184
+ cache_dir=str(data_dir))
185
+ hf_val = load_dataset("1aurent/PatchCamelyon", split="valid",
186
+ cache_dir=str(data_dir))
187
+ hf_test = load_dataset("1aurent/PatchCamelyon", split="test",
188
+ cache_dir=str(data_dir))
189
+
190
+ logger.info("Dataset splits train=%d val=%d test=%d",
191
+ len(hf_train), len(hf_val), len(hf_test))
192
+
193
+ # ── Optional deduplication ────────────────────────────────────────────────
194
+ if getattr(args, "deduplicate", False):
195
+ hf_train = deduplicate_dataset(hf_train, split_name="train")
196
+
197
+ # ── Compute pos_weight from training labels ───────────────────────────────
198
+ # PCam is ~50/50 so pos_weight β‰ˆ 1.0, but we still compute it correctly
199
+ # in case the cached split has slight imbalance.
200
+ all_labels = hf_train["label"] # list of ints, fast column access
201
+ n_benign = all_labels.count(0)
202
+ n_malignant = all_labels.count(1)
203
+ pos_weight = torch.tensor([n_benign / max(n_malignant, 1)], dtype=torch.float32)
204
+
205
+ logger.info("Class balance benign=%d malignant=%d β†’ pos_weight=%.3f",
206
+ n_benign, n_malignant, pos_weight.item())
207
+
208
+ # ── Wrap HuggingFace datasets in PyTorch Dataset objects ──────────────────
209
+ train_transform = build_training_transform()
210
+ eval_transform = build_inference_transform()
211
+
212
+ train_ds = PCamDataset(hf_train, transform=train_transform)
213
+ val_ds = PCamDataset(hf_val, transform=eval_transform)
214
+ test_ds = PCamDataset(hf_test, transform=eval_transform)
215
+
216
+ # ── DataLoaders ───────────────────────────────────────────────────────────
217
+ train_loader = DataLoader(
218
+ train_ds,
219
+ batch_size=args.batch_size,
220
+ shuffle=True,
221
+ num_workers=args.num_workers,
222
+ pin_memory=True,
223
+ drop_last=True,
224
+ )
225
+ val_loader = DataLoader(
226
+ val_ds,
227
+ batch_size=args.batch_size * 2,
228
+ shuffle=False,
229
+ num_workers=args.num_workers,
230
+ pin_memory=True,
231
+ )
232
+ test_loader = DataLoader(
233
+ test_ds,
234
+ batch_size=args.batch_size * 2,
235
+ shuffle=False,
236
+ num_workers=args.num_workers,
237
+ pin_memory=True,
238
+ )
239
+
240
+ return train_loader, val_loader, test_loader, pos_weight
241
+
242
+
243
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
244
+ # β•‘ PCAM PYTORCH DATASET WRAPPER β•‘
245
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
246
+
247
+ class PCamDataset(torch.utils.data.Dataset):
248
+ """
249
+ Wraps a HuggingFace PCam split into a standard PyTorch Dataset.
250
+
251
+ HuggingFace datasets return dicts like {"image": PIL.Image, "label": int}.
252
+ PyTorch DataLoader expects __len__ and __getitem__ returning (tensor, int).
253
+ This class bridges that gap β€” it is the only PCam-specific code in the
254
+ entire project. Everything else (model, inference, API) stays identical.
255
+
256
+ Parameters
257
+ ----------
258
+ hf_dataset : HuggingFace Dataset split
259
+ transform : torchvision transform pipeline to apply to each image
260
+ """
261
+
262
+ def __init__(self, hf_dataset, transform) -> None:
263
+ self.dataset = hf_dataset
264
+ self.transform = transform
265
+
266
+ def __len__(self) -> int:
267
+ return len(self.dataset)
268
+
269
+ def __getitem__(self, idx: int) -> tuple[torch.Tensor, int]:
270
+ sample = self.dataset[idx]
271
+
272
+ # HuggingFace already gives us a PIL Image β€” convert to RGB in case
273
+ # any patches are stored as RGBA or have an unexpected mode
274
+ image = sample["image"].convert("RGB")
275
+ label = int(sample["label"])
276
+
277
+ # Apply the transform pipeline (resize β†’ crop β†’ normalize)
278
+ # This is the same pipeline used by inference.py via ImagePreprocessor
279
+ tensor = self.transform(image) # β†’ (3, 224, 224)
280
+
281
+ return tensor, label
282
+
283
+
284
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
285
+ # β•‘ DEDUPLICATION β•‘
286
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
287
+
288
+ def deduplicate_dataset(hf_dataset, split_name: str = "train"):
289
+ """
290
+ Remove duplicate images from a HuggingFace PCam split using MD5 hashing.
291
+
292
+ The standard PCam dataset contains duplicate images across training samples.
293
+ Duplicates cause the model to memorise specific patches rather than learning
294
+ generalizable tissue patterns β€” directly causing the overfitting seen in
295
+ earlier training runs. This function removes them.
296
+
297
+ The Kaggle PCam competition version has duplicates removed (giving ~220K
298
+ training samples vs the original 262K). This function replicates that
299
+ cleaning step without requiring a Kaggle account.
300
+
301
+ Parameters
302
+ ----------
303
+ hf_dataset : HuggingFace Dataset split
304
+ split_name : for logging only
305
+
306
+ Returns
307
+ -------
308
+ HuggingFace Dataset with duplicate images removed
309
+ """
310
+ import hashlib
311
+
312
+ logger.info("Deduplicating %s split (%d samples) β€” this runs once...",
313
+ split_name, len(hf_dataset))
314
+
315
+ seen_hashes = set()
316
+ keep_indices = []
317
+ batch_size = 500 # process in batches for speed
318
+
319
+ for start in range(0, len(hf_dataset), batch_size):
320
+ end = min(start + batch_size, len(hf_dataset))
321
+ batch = hf_dataset[start:end] # dict of lists β€” fast Arrow column access
322
+ images = batch["image"] # list of PIL Images
323
+
324
+ for i, img in enumerate(images):
325
+ # Hash the raw pixel bytes β€” identical images hash identically
326
+ img_hash = hashlib.md5(img.tobytes()).hexdigest()
327
+ if img_hash not in seen_hashes:
328
+ seen_hashes.add(img_hash)
329
+ keep_indices.append(start + i)
330
+
331
+ if (start // batch_size) % 50 == 0:
332
+ logger.info(" Hashed %d / %d samples...", end, len(hf_dataset))
333
+
334
+ deduped = hf_dataset.select(keep_indices)
335
+ removed = len(hf_dataset) - len(deduped)
336
+ logger.info("Deduplication complete: removed %d duplicates, kept %d samples (%.1f%%)",
337
+ removed, len(deduped), 100 * len(deduped) / len(hf_dataset))
338
+ return deduped
339
+
340
+
341
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
342
+ # β•‘ ONE EPOCH β•‘
343
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
344
+
345
+ def run_epoch(
346
+ model: nn.Module,
347
+ loader: DataLoader,
348
+ criterion: nn.Module,
349
+ optimizer: torch.optim.Optimizer | None,
350
+ device: torch.device,
351
+ phase: str,
352
+ mixup_alpha: float = 0.0,
353
+ scheduler = None, # OneCycleLR β€” stepped per batch
354
+ ) -> tuple[float, float, float]:
355
+ """
356
+ Run one complete pass over a DataLoader.
357
+
358
+ Fix 2 β€” Mixup augmentation
359
+ ──────────────────────────
360
+ When mixup_alpha > 0 and phase == "train", each batch is blended with a
361
+ randomly shuffled version of itself using a Beta(alpha, alpha) weight.
362
+ This forces the model to learn smooth interpolations between patches,
363
+ preventing it from memorising hard boundaries around training examples.
364
+
365
+ Returns (avg_loss, accuracy, sensitivity).
366
+ """
367
+ import numpy as np
368
+
369
+ is_train = (phase == "train")
370
+ use_mixup = is_train and mixup_alpha > 0.0
371
+ model.train() if is_train else model.eval()
372
+
373
+ total_loss = 0.0
374
+ correct = 0
375
+ total = 0
376
+ true_pos = 0
377
+ actual_pos = 0
378
+
379
+ context = torch.enable_grad() if is_train else torch.inference_mode()
380
+
381
+ with context:
382
+ for images, labels in loader:
383
+
384
+ images = images.to(device, non_blocking=True)
385
+ labels = labels.long().to(device, non_blocking=True)
386
+
387
+ # ── Fix 2: Mixup ──────────────────────────────────────────────────
388
+ if use_mixup:
389
+ # Sample mixing weight from Beta distribution
390
+ lam = float(np.random.beta(mixup_alpha, mixup_alpha))
391
+ # Randomly shuffle the batch to get mixing partners
392
+ idx = torch.randperm(images.size(0), device=device)
393
+ mixed_images = lam * images + (1.0 - lam) * images[idx]
394
+ labels_b = labels[idx]
395
+
396
+ output = model(mixed_images)
397
+ # Mixed loss: weighted combination of both label sets
398
+ loss = (lam * criterion(output["logits"], labels) +
399
+ (1 - lam) * criterion(output["logits"], labels_b))
400
+ else:
401
+ output = model(images)
402
+ loss = criterion(output["logits"], labels)
403
+
404
+ # ���─ Backward pass (training only) ─────────────────────────────────
405
+ if is_train:
406
+ optimizer.zero_grad()
407
+ loss.backward()
408
+ nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
409
+ optimizer.step()
410
+ if scheduler is not None:
411
+ scheduler.step() # OneCycleLR steps every batch
412
+
413
+ # ── Accumulate metrics (always on original labels) ────────────────
414
+ preds = output["logits"].argmax(dim=1)
415
+ correct += (preds == labels).sum().item()
416
+ total += labels.size(0)
417
+ total_loss += loss.item() * labels.size(0)
418
+
419
+ malignant_mask = (labels == 1)
420
+ true_pos += (preds[malignant_mask] == 1).sum().item()
421
+ actual_pos += malignant_mask.sum().item()
422
+
423
+ avg_loss = total_loss / max(total, 1)
424
+ accuracy = correct / max(total, 1)
425
+ sensitivity = true_pos / max(actual_pos, 1)
426
+
427
+ return avg_loss, accuracy, sensitivity
428
+
429
+
430
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
431
+ # β•‘ CHECKPOINT SAVE β•‘
432
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
433
+
434
+ def save_checkpoint(
435
+ model: nn.Module,
436
+ optimizer: torch.optim.Optimizer,
437
+ epoch: int,
438
+ val_loss: float,
439
+ val_acc: float,
440
+ path: str | Path,
441
+ ) -> None:
442
+ """
443
+ Save model weights + training metadata to disk.
444
+
445
+ The saved dict uses the key "state_dict" β€” exactly what
446
+ inference.py's _load_weights() looks for when loading weights.
447
+ The extra keys (optimizer, epoch, val_loss, val_acc) are ignored
448
+ by inference.py but useful if you want to resume training later.
449
+ """
450
+ Path(path).parent.mkdir(parents=True, exist_ok=True)
451
+ torch.save(
452
+ {
453
+ "state_dict": model.state_dict(), # ← inference.py reads this
454
+ "optimizer": optimizer.state_dict(),
455
+ "epoch": epoch,
456
+ "val_loss": val_loss,
457
+ "val_acc": val_acc,
458
+ },
459
+ path,
460
+ )
461
+ logger.info(" βœ“ Best checkpoint saved β†’ %s", path)
462
+
463
+
464
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
465
+ # β•‘ CSV LOGGER β•‘
466
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
467
+
468
+ class CSVLogger:
469
+ """Writes one row per epoch to a CSV file so you can plot training curves."""
470
+
471
+ FIELDS = [
472
+ "epoch", "train_loss", "train_acc", "train_sens",
473
+ "val_loss", "val_acc", "val_sens", "lr", "epoch_time_s",
474
+ ]
475
+
476
+ def __init__(self, path: str | Path) -> None:
477
+ self.path = Path(path)
478
+ # Write header on creation β€” overwrites any previous log file
479
+ with open(self.path, "w", newline="") as f:
480
+ csv.DictWriter(f, fieldnames=self.FIELDS).writeheader()
481
+
482
+ def log(self, row: dict) -> None:
483
+ with open(self.path, "a", newline="") as f:
484
+ writer = csv.DictWriter(f, fieldnames=self.FIELDS)
485
+ writer.writerow(
486
+ {k: round(v, 6) if isinstance(v, float) else v
487
+ for k, v in row.items()}
488
+ )
489
+
490
+
491
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
492
+ # β•‘ DEVICE RESOLUTION β•‘
493
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
494
+
495
+ def resolve_device() -> torch.device:
496
+ """Auto-detect best available device: CUDA β†’ Apple MPS β†’ CPU."""
497
+ if torch.cuda.is_available():
498
+ return torch.device("cuda")
499
+ if torch.backends.mps.is_available():
500
+ return torch.device("mps")
501
+ return torch.device("cpu")
502
+
503
+
504
+ # ╔════════════════════════════════════���═════════════════════════════════════════╗
505
+ # β•‘ MAIN β•‘
506
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
507
+
508
+ def main() -> None:
509
+ args = parse_args()
510
+ device = resolve_device()
511
+
512
+ # ── Reproducibility ───────────────────────────────────────────────────────
513
+ # Setting the seed ensures you get the same weight initializations,
514
+ # data shuffling order, and dropout masks every run. Makes results comparable.
515
+ torch.manual_seed(args.seed)
516
+ if device.type == "cuda":
517
+ torch.cuda.manual_seed_all(args.seed)
518
+
519
+ logger.info("═" * 58)
520
+ logger.info(" Breast Cancer Classifier β€” Training Run (PCam)")
521
+ logger.info(" Device : %s", device)
522
+ logger.info(" Epochs : %d", args.epochs)
523
+ logger.info(" Batch size : %d", args.batch_size)
524
+ logger.info(" LR : %g", args.lr)
525
+ logger.info(" Freeze : %s", args.freeze_backbone)
526
+ logger.info(" Mixup alpha : %.2f", args.mixup_alpha)
527
+ logger.info(" Lbl smooth : %.2f", args.label_smoothing)
528
+ logger.info(" Max LR : %g", args.max_lr)
529
+ logger.info(" Deduplicate : %s", args.deduplicate)
530
+ logger.info(" Output : %s", args.weights_out)
531
+ logger.info("═" * 58)
532
+
533
+ # ── Step 1 β€” Load data ────────────────────────────────────────────────────
534
+ train_loader, val_loader, test_loader, pos_weight = get_dataloaders(args)
535
+
536
+ # ── Step 2 β€” Build model ──────────────────────────────────────────────────
537
+ # Uses BreastCancerClassifier from model/model.py β€” the same class
538
+ # that inference.py uses at prediction time.
539
+ model = BreastCancerClassifier(
540
+ pretrained=True, # start from ImageNet weights
541
+ freeze_backbone=args.freeze_backbone,
542
+ dropout_rate=args.dropout,
543
+ ).to(device)
544
+
545
+ trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
546
+ total = sum(p.numel() for p in model.parameters())
547
+ logger.info("Parameters trainable=%s / total=%s",
548
+ f"{trainable:,}", f"{total:,}")
549
+
550
+ # ── Step 3 β€” Loss function ────────────────────────────────────────────────
551
+ # CrossEntropyLoss with class weights.
552
+ # [1.0, pos_weight] = [benign weight, malignant weight]
553
+ # The malignant class gets upweighted because:
554
+ # (a) it's the minority class
555
+ # (b) missing a malignant case is clinically much worse than a false alarm
556
+ criterion = nn.CrossEntropyLoss(
557
+ weight = torch.tensor([1.0, pos_weight.item()], device=device),
558
+ label_smoothing = args.label_smoothing, # Fix 3: label smoothing
559
+ )
560
+
561
+ # ── Step 4 β€” Optimizer ────────────────────────────────────────────────────
562
+ # Adam is used because it adapts the learning rate per parameter β€”
563
+ # well suited for fine-tuning pretrained networks.
564
+ # filter(...) skips frozen backbone parameters so they receive no updates.
565
+ optimizer = Adam(
566
+ filter(lambda p: p.requires_grad, model.parameters()),
567
+ lr=args.lr,
568
+ weight_decay=args.weight_decay,
569
+ )
570
+
571
+ # ── Step 5 β€” LR Scheduler ─────────────────────────────────────────────────
572
+ # OneCycleLR ramps the LR up to max_lr then back down over all epochs.
573
+ # This breaks out of bad local minima early (ramp up) then converges
574
+ # precisely (ramp down). Proven to reach 97%+ on PCam in published work.
575
+ # Must be stepped every BATCH β€” see run_epoch() for the scheduler.step() call.
576
+ scheduler = OneCycleLR(
577
+ optimizer,
578
+ max_lr = args.max_lr,
579
+ steps_per_epoch = len(train_loader), # batches per epoch
580
+ epochs = args.epochs,
581
+ pct_start = 0.3, # 30% of training ramps up
582
+ anneal_strategy = "cos",
583
+ )
584
+
585
+ # ── Training state ────────────────────────────────────────────────────────
586
+ csv_logger = CSVLogger(args.log_out)
587
+ best_val_sens = 0.0 # track sensitivity, not loss
588
+ epochs_no_improve = 0
589
+
590
+ # ═══════════════════════════════════════════════════════════════════════════
591
+ # EPOCH LOOP
592
+ # ═══════════════════════════════════════════════════════════════════════════
593
+ for epoch in range(1, args.epochs + 1):
594
+ t0 = time.time()
595
+
596
+ # ── Train for one full epoch ───────────────────────────────────────────
597
+ train_loss, train_acc, train_sens = run_epoch(
598
+ model, train_loader, criterion, optimizer, device,
599
+ phase="train", mixup_alpha=args.mixup_alpha,
600
+ scheduler=scheduler,
601
+ )
602
+
603
+ # ── Validate (no weight updates) ──────────────────────────────────────
604
+ val_loss, val_acc, val_sens = run_epoch(
605
+ model, val_loader, criterion, None, device, phase="val"
606
+ )
607
+
608
+ elapsed = time.time() - t0
609
+ current_lr = optimizer.param_groups[0]["lr"]
610
+
611
+ # ── Print epoch summary ───────────────────────────────────────────────
612
+ logger.info(
613
+ "Epoch %2d/%d β”‚ "
614
+ "train loss=%.4f acc=%.3f sens=%.3f β”‚ "
615
+ "val loss=%.4f acc=%.3f sens=%.3f β”‚ "
616
+ "lr=%.1e time=%.1fs",
617
+ epoch, args.epochs,
618
+ train_loss, train_acc, train_sens,
619
+ val_loss, val_acc, val_sens,
620
+ current_lr, elapsed,
621
+ )
622
+
623
+ # ── Log to CSV ────────────────────────────────────────────────────────
624
+ csv_logger.log({
625
+ "epoch": epoch,
626
+ "train_loss": train_loss,
627
+ "train_acc": train_acc,
628
+ "train_sens": train_sens,
629
+ "val_loss": val_loss,
630
+ "val_acc": val_acc,
631
+ "val_sens": val_sens,
632
+ "lr": current_lr,
633
+ "epoch_time_s": elapsed,
634
+ })
635
+
636
+ # ── Save if this is the best model so far ─────────────────────────────
637
+ if val_sens > best_val_sens:
638
+ best_val_sens = val_sens
639
+ epochs_no_improve = 0
640
+ save_checkpoint(
641
+ model, optimizer, epoch,
642
+ val_loss, val_acc,
643
+ path=args.weights_out,
644
+ )
645
+ logger.info(" β˜… New best sensitivity: %.3f", best_val_sens)
646
+ else:
647
+ epochs_no_improve += 1
648
+ logger.info(" No improvement (%d/%d before early stop)",
649
+ epochs_no_improve, args.early_stop or args.epochs)
650
+
651
+ # ── Early stopping ────────────────────────────────────────────────────
652
+ if args.early_stop > 0 and epochs_no_improve >= args.early_stop:
653
+ logger.info(
654
+ "Early stopping: no improvement for %d consecutive epochs.",
655
+ args.early_stop,
656
+ )
657
+ break
658
+
659
+ # ═══════════════════════════════════════════════════════════════════════════
660
+ # FINAL TEST EVALUATION
661
+ # Always uses the BEST checkpoint, not the final epoch's weights
662
+ # ═══════════════════════════════════════════════════════════════════════════
663
+ logger.info("─" * 58)
664
+ logger.info("Reloading best checkpoint for final test evaluation...")
665
+
666
+ best_ckpt = torch.load(args.weights_out, map_location=device)
667
+ model.load_state_dict(best_ckpt["state_dict"])
668
+
669
+ test_loss, test_acc, test_sens = run_epoch(
670
+ model, test_loader, criterion, None, device, phase="test"
671
+ )
672
+
673
+ logger.info("═" * 58)
674
+ logger.info(" FINAL TEST RESULTS (from epoch %d checkpoint)",
675
+ best_ckpt["epoch"])
676
+ logger.info(" Loss : %.4f", test_loss)
677
+ logger.info(" Accuracy : %.1f%%", test_acc * 100)
678
+ logger.info(" Sensitivity : %.1f%%", test_sens * 100)
679
+ logger.info("─" * 58)
680
+ logger.info(" Weights saved : %s", args.weights_out)
681
+ logger.info(" Training log : %s", args.log_out)
682
+ logger.info("═" * 58)
683
+ logger.info(" Run inference:")
684
+ logger.info(" from model import BreastCancerInferencePipeline")
685
+ logger.info(" p = BreastCancerInferencePipeline('%s')", args.weights_out)
686
+ logger.info(" p.predict('slide.png')")
687
+ logger.info("═" * 58)
688
+
689
+
690
+ # ── Entry point ───────────────────────────────────────────────────────────────
691
+ if __name__ == "__main__":
692
+ main()
train_mammogram.py ADDED
@@ -0,0 +1,838 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ train_mammogram.py
3
+ ───────────────────
4
+ Standalone EfficientNet-B4 mammogram training script.
5
+ Runs on any Linux machine with a GPU β€” no Modal, no cloud API needed.
6
+
7
+ Designed for vast.ai RTX 3090 (24GB) or any GPU with 16GB+ VRAM.
8
+
9
+ Usage
10
+ ─────
11
+ # Full training
12
+ python train_mammogram.py
13
+
14
+ # Debug mode (5% data, 2 epochs)
15
+ python train_mammogram.py --debug
16
+
17
+ # Resume Phase 2 from Phase 1 checkpoint
18
+ python train_mammogram.py --skip-phase1
19
+
20
+ # Custom paths
21
+ python train_mammogram.py --rsna-dir /data/rsna --vindr-dir /data/vindr
22
+
23
+ Setup on vast.ai
24
+ ─────────────────
25
+ # 1. Rent instance
26
+ # GPU: RTX 3090 (24GB) ~$0.30/hr
27
+ # Disk: 500GB
28
+ # Image: pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime
29
+
30
+ # 2. SSH in and install dependencies
31
+ pip install pydicom pylibjpeg python-gdcm kaggle scikit-learn tqdm pandas
32
+
33
+ # 3. Set Kaggle credentials
34
+ mkdir -p ~/.kaggle
35
+ echo '{"username":"relixsx","key":"your_key"}' > ~/.kaggle/kaggle.json
36
+ chmod 600 ~/.kaggle/kaggle.json
37
+
38
+ # 4. Run in tmux (survives disconnection)
39
+ tmux new -s training
40
+ python train_mammogram.py
41
+ # Ctrl+B then D to detach β€” training keeps running
42
+ # tmux attach -t training to reconnect
43
+
44
+ # 5. When done, download weights
45
+ scp -P PORT root@IP:/workspace/outputs/mammogram_weights.pth ./model/
46
+ scp -P PORT root@IP:/workspace/outputs/mammogram_results.json ./
47
+ """
48
+
49
+ from __future__ import annotations
50
+
51
+ import argparse
52
+ import csv
53
+ import json
54
+ import logging
55
+ import os
56
+ import subprocess
57
+ import sys
58
+ import time
59
+ import warnings
60
+ from pathlib import Path
61
+
62
+ # Suppress pydicom deprecation warnings
63
+ warnings.filterwarnings("ignore", category=DeprecationWarning)
64
+ os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
65
+
66
+ import numpy as np
67
+ import pandas as pd
68
+ import torch
69
+ import torch.nn as nn
70
+ import torch.nn.functional as F
71
+ from PIL import Image
72
+ from sklearn.metrics import f1_score, roc_auc_score, confusion_matrix, roc_curve
73
+ from sklearn.model_selection import train_test_split
74
+ from torch.optim import AdamW
75
+ from torch.optim.lr_scheduler import OneCycleLR
76
+ from torch.utils.data import Dataset, DataLoader
77
+ from torchvision import models, transforms
78
+ from torchvision.models import EfficientNet_B4_Weights
79
+
80
+ # ── Logging ────────────────────────────────────────────────────────────────────
81
+ logging.basicConfig(
82
+ level = logging.INFO,
83
+ format = "%(asctime)s %(levelname)-8s %(message)s",
84
+ datefmt = "%H:%M:%S",
85
+ handlers=[
86
+ logging.StreamHandler(sys.stdout),
87
+ logging.FileHandler("training.log"),
88
+ ],
89
+ )
90
+ logger = logging.getLogger(__name__)
91
+
92
+
93
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
94
+ # β•‘ MODEL DEFINITIONS β•‘
95
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
96
+
97
+ class FocalLoss(nn.Module):
98
+ def __init__(self, alpha=0.25, gamma=2.0, pos_weight=None, label_smoothing=0.1):
99
+ super().__init__()
100
+ self.alpha = alpha
101
+ self.gamma = gamma
102
+ self.pos_weight = pos_weight
103
+ self.label_smoothing = label_smoothing
104
+
105
+ def forward(self, logits, targets):
106
+ ce = F.cross_entropy(
107
+ logits, targets,
108
+ weight = self.pos_weight,
109
+ label_smoothing = self.label_smoothing,
110
+ reduction = "none",
111
+ )
112
+ pt = torch.exp(-ce)
113
+ return (self.alpha * (1 - pt) ** self.gamma * ce).mean()
114
+
115
+
116
+ class ViewAttentionFusion(nn.Module):
117
+ """Soft-attention fusion of CC and MLO view features."""
118
+
119
+ def __init__(self, dim=1792, dropout=0.2):
120
+ super().__init__()
121
+ self.gate = nn.Sequential(
122
+ nn.Linear(dim * 2, dim // 4),
123
+ nn.ReLU(inplace=True),
124
+ nn.Dropout(dropout),
125
+ nn.Linear(dim // 4, 2),
126
+ )
127
+ self.residual_w = nn.Parameter(torch.tensor(0.5))
128
+
129
+ def forward(self, cc_feat, mlo_feat):
130
+ weights = torch.softmax(
131
+ self.gate(torch.cat([cc_feat, mlo_feat], dim=-1)), dim=-1
132
+ )
133
+ attended = weights[:, 0:1] * cc_feat + weights[:, 1:2] * mlo_feat
134
+ simple = 0.5 * cc_feat + 0.5 * mlo_feat
135
+ w = torch.sigmoid(self.residual_w)
136
+ return w * attended + (1 - w) * simple
137
+
138
+
139
+ class MultiViewMammogramClassifier(nn.Module):
140
+ """Siamese EfficientNet-B4 with attention-based CC+MLO view fusion."""
141
+
142
+ def __init__(self, pretrained=True, freeze_backbone=False, dropout_rate=0.4):
143
+ super().__init__()
144
+ weights = EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None
145
+ bb = models.efficientnet_b4(weights=weights)
146
+ self.features = bb.features
147
+ self.avgpool = bb.avgpool
148
+
149
+ if freeze_backbone:
150
+ for p in self.features.parameters():
151
+ p.requires_grad = False
152
+
153
+ self.fusion = ViewAttentionFusion(dim=1792)
154
+ self.classifier = nn.Sequential(
155
+ nn.BatchNorm1d(1792),
156
+ nn.Dropout(dropout_rate),
157
+ nn.Linear(1792, 512),
158
+ nn.ReLU(inplace=True),
159
+ nn.BatchNorm1d(512),
160
+ nn.Dropout(dropout_rate * 0.75),
161
+ nn.Linear(512, 2),
162
+ )
163
+
164
+ def encode(self, x):
165
+ return torch.flatten(self.avgpool(self.features(x)), 1)
166
+
167
+ def forward(self, cc, mlo):
168
+ logits = self.classifier(self.fusion(self.encode(cc), self.encode(mlo)))
169
+ return {"logits": logits, "probs": torch.softmax(logits, 1)}
170
+
171
+ def forward_single(self, x):
172
+ return self.forward(x, x)
173
+
174
+
175
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
176
+ # β•‘ DICOM LOADER β•‘
177
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
178
+
179
+ def dicom_to_rgb(path: str) -> Image.Image:
180
+ import pydicom
181
+ from pydicom.pixels import apply_voi_lut
182
+
183
+ dcm = pydicom.dcmread(path)
184
+ try:
185
+ arr = apply_voi_lut(dcm.pixel_array, dcm)
186
+ except Exception:
187
+ arr = dcm.pixel_array.astype(np.float32)
188
+
189
+ arr = arr.astype(np.float32)
190
+ mn, mx = arr.min(), arr.max()
191
+ if mx > mn:
192
+ arr = (arr - mn) / (mx - mn) * 255.0
193
+ arr = arr.astype(np.uint8)
194
+
195
+ if getattr(dcm, "PhotometricInterpretation", "") == "MONOCHROME1":
196
+ arr = 255 - arr
197
+ if arr.ndim == 2:
198
+ arr = np.stack([arr, arr, arr], axis=-1)
199
+ return Image.fromarray(arr, mode="RGB")
200
+
201
+
202
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
203
+ # β•‘ METRICS β•‘
204
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
205
+
206
+ def compute_metrics(y_true, y_prob, threshold=0.5):
207
+ y_true = np.array(y_true)
208
+ y_prob = np.array(y_prob)
209
+ y_pred = (y_prob >= threshold).astype(int)
210
+ auc = roc_auc_score(y_true, y_prob) if len(set(y_true)) > 1 else 0.0
211
+ cm = confusion_matrix(y_true, y_pred)
212
+ tn, fp, fn, tp = cm.ravel() if cm.size == 4 else (0, 0, 0, 0)
213
+ return {
214
+ "auc": round(float(auc), 4),
215
+ "sensitivity": round(tp / max(tp + fn, 1), 4),
216
+ "specificity": round(tn / max(tn + fp, 1), 4),
217
+ "ppv": round(tp / max(tp + fp, 1), 4),
218
+ "npv": round(tn / max(tn + fn, 1), 4),
219
+ "accuracy": round((tp + tn) / max(len(y_true), 1), 4),
220
+ "f1": round(float(f1_score(y_true, y_pred, zero_division=0)), 4),
221
+ "n_pos": int(y_true.sum()),
222
+ "n_neg": int((1 - y_true).sum()),
223
+ }
224
+
225
+
226
+ def youden_threshold(y_true, y_prob):
227
+ fpr, tpr, thr = roc_curve(y_true, y_prob)
228
+ return float(thr[np.argmax(tpr - fpr)])
229
+
230
+
231
+ def bootstrap_auc_ci(y_true, y_prob, n=1000):
232
+ rng = np.random.default_rng(42)
233
+ aucs = []
234
+ y_true, y_prob = np.array(y_true), np.array(y_prob)
235
+ for _ in range(n):
236
+ idx = rng.integers(0, len(y_true), len(y_true))
237
+ yt, yp = y_true[idx], y_prob[idx]
238
+ if len(set(yt)) > 1:
239
+ aucs.append(roc_auc_score(yt, yp))
240
+ return (
241
+ round(float(np.percentile(aucs, 2.5)), 4),
242
+ round(float(np.percentile(aucs, 97.5)), 4),
243
+ )
244
+
245
+
246
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
247
+ # β•‘ DATA DOWNLOAD β•‘
248
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
249
+
250
+ def download_rsna(rsna_dir: Path):
251
+ if (rsna_dir / "train.csv").exists():
252
+ logger.info("RSNA 2022 found in cache.")
253
+ return
254
+
255
+ logger.info("Downloading RSNA 2022 (~300 GB) to /tmp then unzipping...")
256
+ os.makedirs(str(rsna_dir), exist_ok=True)
257
+
258
+ subprocess.run([
259
+ "kaggle", "competitions", "download",
260
+ "-c", "rsna-breast-cancer-detection",
261
+ "-p", "/tmp",
262
+ ], check=True)
263
+
264
+ import glob
265
+ zips = glob.glob("/tmp/*.zip")
266
+ tmp_zip = zips[0] if zips else "/tmp/rsna-breast-cancer-detection.zip"
267
+
268
+ logger.info("Unzipping RSNA to %s ...", rsna_dir)
269
+ subprocess.run(["unzip", "-q", tmp_zip, "-d", str(rsna_dir)], check=True)
270
+ os.remove(tmp_zip)
271
+ logger.info("RSNA ready.")
272
+
273
+
274
+ def download_vindr(vindr_dir: Path, physionet_user: str, physionet_pass: str):
275
+ if (vindr_dir / "breast-level_annotations.csv").exists():
276
+ logger.info("VinDr-Mammo found in cache.")
277
+ return
278
+
279
+ logger.info("Downloading VinDr-Mammo from PhysioNet (~70 GB)...")
280
+ os.makedirs(str(vindr_dir), exist_ok=True)
281
+
282
+ # Download CSV annotations first (fast)
283
+ for csv_file in ["breast-level_annotations.csv", "finding_annotations.csv"]:
284
+ subprocess.run([
285
+ "wget", "-N", "-c", "-q",
286
+ f"--user={physionet_user}",
287
+ f"--password={physionet_pass}",
288
+ "-P", str(vindr_dir),
289
+ f"https://physionet.org/files/vindr-mammo/1.0.0/{csv_file}",
290
+ ], check=True)
291
+
292
+ # Download DICOM images β€” skip HTML crawling
293
+ subprocess.run([
294
+ "wget", "-r", "-N", "-c", "-np", "-q",
295
+ "--accept=*.dicom,*.dcm",
296
+ "--reject=*.html,*.php,index*,robots*",
297
+ f"--user={physionet_user}",
298
+ f"--password={physionet_pass}",
299
+ "-P", str(vindr_dir),
300
+ "https://physionet.org/files/vindr-mammo/1.0.0/images/",
301
+ ], check=True)
302
+
303
+ # Move from nested wget path
304
+ import shutil
305
+ nested = vindr_dir / "physionet.org" / "files" / "vindr-mammo" / "1.0.0"
306
+ if nested.exists():
307
+ for item in nested.iterdir():
308
+ shutil.move(str(item), str(vindr_dir / item.name))
309
+ shutil.rmtree(str(vindr_dir / "physionet.org"))
310
+
311
+ logger.info("VinDr-Mammo ready.")
312
+
313
+
314
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
315
+ # β•‘ DATASETS β•‘
316
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
317
+
318
+ class MultiViewDataset(Dataset):
319
+ def __init__(self, cases_df, img_dir, transform, size):
320
+ self.cases = cases_df.reset_index(drop=True)
321
+ self.img_dir = Path(img_dir)
322
+ self.transform = transform
323
+ self.size = size
324
+
325
+ def _load(self, patient_id, image_id):
326
+ path = self.img_dir / "train_images" / str(patient_id) / f"{image_id}.dcm"
327
+ try:
328
+ return dicom_to_rgb(str(path))
329
+ except Exception as e:
330
+ logger.warning("Load error %s: %s", path.name, e)
331
+ return Image.new("RGB", (self.size, self.size), 0)
332
+
333
+ def __len__(self):
334
+ return len(self.cases)
335
+
336
+ def __getitem__(self, idx):
337
+ row = self.cases.iloc[idx]
338
+ return (
339
+ self.transform(self._load(row["patient_id"], row["cc_img"])),
340
+ self.transform(self._load(row["patient_id"], row["mlo_img"])),
341
+ int(row["label"]),
342
+ )
343
+
344
+
345
+ class VinDrDataset(Dataset):
346
+ def __init__(self, df, img_dir, transform):
347
+ self.df = df.reset_index(drop=True)
348
+ self.img_dir = Path(img_dir)
349
+ self.transform = transform
350
+
351
+ def __len__(self):
352
+ return len(self.df)
353
+
354
+ def __getitem__(self, idx):
355
+ row = self.df.iloc[idx]
356
+ study = str(row.get("study_id", ""))
357
+ img_id = str(row.get("image_id", ""))
358
+ label = int(row["label"])
359
+ path = None
360
+ for ext in [".dicom", ".dcm"]:
361
+ p = self.img_dir / "images" / study / f"{img_id}{ext}"
362
+ if p.exists():
363
+ path = p
364
+ break
365
+ try:
366
+ img = dicom_to_rgb(str(path)) if path else Image.new("RGB", (512, 512), 0)
367
+ except Exception:
368
+ img = Image.new("RGB", (512, 512), 0)
369
+ return self.transform(img), label
370
+
371
+
372
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
373
+ # β•‘ TRANSFORMS β•‘
374
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•οΏ½οΏ½β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
375
+
376
+ IMGNET_MEAN = [0.485, 0.456, 0.406]
377
+ IMGNET_STD = [0.229, 0.224, 0.225]
378
+
379
+
380
+ def make_train_tf(size):
381
+ return transforms.Compose([
382
+ transforms.Resize((size, size)),
383
+ transforms.RandomHorizontalFlip(),
384
+ transforms.RandomVerticalFlip(p=0.2),
385
+ transforms.RandomRotation(10),
386
+ transforms.ColorJitter(brightness=0.15, contrast=0.15),
387
+ transforms.RandomAffine(0, translate=(0.05, 0.05), scale=(0.95, 1.05)),
388
+ transforms.ToTensor(),
389
+ transforms.Normalize(IMGNET_MEAN, IMGNET_STD),
390
+ ])
391
+
392
+
393
+ def make_val_tf(size=256):
394
+ return transforms.Compose([
395
+ transforms.Resize((size, size)),
396
+ transforms.ToTensor(),
397
+ transforms.Normalize(IMGNET_MEAN, IMGNET_STD),
398
+ ])
399
+
400
+
401
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
402
+ # β•‘ TRAINING β•‘
403
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
404
+
405
+ def run_train_epoch(model, loader, criterion, optimizer, scheduler, scaler, device):
406
+ model.train()
407
+ total_loss = tp = total = pos = 0
408
+
409
+ for cc_imgs, mlo_imgs, labels in loader:
410
+ cc_imgs = cc_imgs.to(device, non_blocking=True)
411
+ mlo_imgs = mlo_imgs.to(device, non_blocking=True)
412
+ labels = labels.long().to(device, non_blocking=True)
413
+
414
+ with torch.autocast(device_type="cuda", dtype=torch.float16):
415
+ out = model(cc_imgs, mlo_imgs)
416
+ loss = criterion(out["logits"], labels)
417
+
418
+ optimizer.zero_grad()
419
+ scaler.scale(loss).backward()
420
+ scaler.unscale_(optimizer)
421
+ nn.utils.clip_grad_norm_(model.parameters(), 1.0)
422
+ scaler.step(optimizer)
423
+ scaler.update()
424
+ scheduler.step()
425
+
426
+ preds = out["logits"].argmax(1)
427
+ total += labels.size(0)
428
+ total_loss += loss.item() * labels.size(0)
429
+ mask = (labels == 1)
430
+ tp += (preds[mask] == 1).sum().item()
431
+ pos += mask.sum().item()
432
+
433
+ return total_loss / max(total, 1), tp / max(pos, 1)
434
+
435
+
436
+ def run_eval_epoch(model, loader, device, is_multiview=True):
437
+ model.eval()
438
+ all_probs, all_labels = [], []
439
+
440
+ with torch.inference_mode():
441
+ for batch in loader:
442
+ if is_multiview:
443
+ cc_imgs, mlo_imgs, labels = batch
444
+ out = model(cc_imgs.to(device), mlo_imgs.to(device))
445
+ else:
446
+ imgs, labels = batch
447
+ out = model.forward_single(imgs.to(device))
448
+ all_probs.extend(out["probs"][:, 1].cpu().numpy().tolist())
449
+ all_labels.extend(labels.numpy().tolist())
450
+
451
+ return all_labels, all_probs
452
+
453
+
454
+ def save_checkpoint(model, optimizer, epoch, metrics, path):
455
+ torch.save({
456
+ "epoch": epoch,
457
+ "state_dict": model.state_dict(),
458
+ "optimizer": optimizer.state_dict(),
459
+ "metrics": metrics,
460
+ }, path)
461
+ logger.info("Checkpoint saved β†’ %s", path)
462
+
463
+
464
+ # ╔══════════════════════════════════════════════════════════════════════════════╗
465
+ # β•‘ MAIN β•‘
466
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
467
+
468
+ def main():
469
+ parser = argparse.ArgumentParser(description="MedAI Mammogram Training")
470
+ parser.add_argument("--rsna-dir", default="/workspace/data/rsna")
471
+ parser.add_argument("--vindr-dir", default="/workspace/data/vindr")
472
+ parser.add_argument("--output-dir", default="/workspace/outputs")
473
+ parser.add_argument("--physionet-user", default=os.getenv("PHYSIONET_USERNAME", ""))
474
+ parser.add_argument("--physionet-pass", default=os.getenv("PHYSIONET_PASSWORD", ""))
475
+ parser.add_argument("--phase1-epochs", type=int, default=5)
476
+ parser.add_argument("--phase2-epochs", type=int, default=15)
477
+ parser.add_argument("--batch-size", type=int, default=16)
478
+ parser.add_argument("--phase1-lr", type=float, default=3e-4)
479
+ parser.add_argument("--phase2-lr", type=float, default=5e-5)
480
+ parser.add_argument("--max-lr", type=float, default=1e-3)
481
+ parser.add_argument("--focal-alpha", type=float, default=0.25)
482
+ parser.add_argument("--focal-gamma", type=float, default=2.0)
483
+ parser.add_argument("--num-workers", type=int, default=4)
484
+ parser.add_argument("--seed", type=int, default=42)
485
+ parser.add_argument("--skip-phase1", action="store_true")
486
+ parser.add_argument("--debug", action="store_true",
487
+ help="Use 5% of data and 2 epochs for testing")
488
+ args = parser.parse_args()
489
+
490
+ # ── Setup ─────────────────────────────────────────────────────────────────
491
+ import random
492
+ random.seed(args.seed)
493
+ np.random.seed(args.seed)
494
+ torch.manual_seed(args.seed)
495
+
496
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
497
+ logger.info("Device: %s", device)
498
+ if torch.cuda.is_available():
499
+ torch.cuda.manual_seed_all(args.seed)
500
+ logger.info("GPU : %s", torch.cuda.get_device_name(0))
501
+ logger.info("VRAM: %.1f GB",
502
+ torch.cuda.get_device_properties(0).total_memory / 1e9)
503
+
504
+ rsna_dir = Path(args.rsna_dir)
505
+ vindr_dir = Path(args.vindr_dir)
506
+ output_dir = Path(args.output_dir)
507
+ output_dir.mkdir(parents=True, exist_ok=True)
508
+
509
+ logger.info("=" * 70)
510
+ logger.info(" EfficientNet-B4 Mammogram Training β€” 5 Innovations")
511
+ logger.info(" Train: RSNA 2022 | External val: VinDr-Mammo")
512
+ logger.info(" Debug: %s", args.debug)
513
+ logger.info("=" * 70)
514
+
515
+ # ── Download data ─────────────────────────────────────────────────────────
516
+ download_rsna(rsna_dir)
517
+ if args.physionet_user:
518
+ download_vindr(vindr_dir, args.physionet_user, args.physionet_pass)
519
+ else:
520
+ logger.warning("PHYSIONET_USERNAME not set β€” skipping VinDr download.")
521
+
522
+ # ── Load RSNA labels ──────────────────────────────────────────────────────
523
+ rsna_df = pd.read_csv(rsna_dir / "train.csv")
524
+ logger.info("RSNA: %d images | cancer: %.1f%%",
525
+ len(rsna_df), 100 * rsna_df["cancer"].mean())
526
+
527
+ if args.debug:
528
+ rsna_df = rsna_df.sample(frac=0.05, random_state=args.seed).reset_index(drop=True)
529
+ args.phase1_epochs = min(2, args.phase1_epochs)
530
+ args.phase2_epochs = min(2, args.phase2_epochs)
531
+ logger.info("DEBUG: %d samples", len(rsna_df))
532
+
533
+ # ── Build multi-view pairs ────────────────────────────────────────────────
534
+ logger.info("Building multi-view pairs...")
535
+ cases = []
536
+ if "view" in rsna_df.columns and "laterality" in rsna_df.columns:
537
+ for (pid, lat), grp in rsna_df.groupby(["patient_id", "laterality"]):
538
+ cc = grp[grp["view"] == "CC"]
539
+ mlo = grp[grp["view"] == "MLO"]
540
+ lbl = int(grp["cancer"].max())
541
+ if len(cc) > 0 and len(mlo) > 0:
542
+ cases.append({"patient_id": pid, "laterality": lat,
543
+ "cc_img": cc.iloc[0]["image_id"],
544
+ "mlo_img": mlo.iloc[0]["image_id"],
545
+ "label": lbl})
546
+ else:
547
+ row = grp.iloc[0]
548
+ cases.append({"patient_id": pid, "laterality": lat,
549
+ "cc_img": row["image_id"],
550
+ "mlo_img": row["image_id"],
551
+ "label": lbl})
552
+ else:
553
+ for _, row in rsna_df.iterrows():
554
+ cases.append({"patient_id": row["patient_id"],
555
+ "laterality": row.get("laterality", "L"),
556
+ "cc_img": row["image_id"],
557
+ "mlo_img": row["image_id"],
558
+ "label": int(row["cancer"])})
559
+
560
+ cases_df = pd.DataFrame(cases)
561
+ train_df, val_df = train_test_split(
562
+ cases_df, test_size=0.15, stratify=cases_df["label"], random_state=args.seed
563
+ )
564
+ logger.info("Train: %d | Val: %d", len(train_df), len(val_df))
565
+
566
+ # ── Load VinDr labels ─────────────────────────────────────────────────────
567
+ vindr_ext = pd.DataFrame()
568
+ vindr_csv = vindr_dir / "breast-level_annotations.csv"
569
+ if vindr_csv.exists():
570
+ vindr_df = pd.read_csv(vindr_csv)
571
+ bc = "breast_birads" if "breast_birads" in vindr_df.columns else "birads"
572
+ vindr_df["label"] = vindr_df[bc].map(
573
+ {"BI-RADS 1": 0, "BI-RADS 2": 0, "BI-RADS 4": 1, "BI-RADS 5": 1,
574
+ 1: 0, 2: 0, 4: 1, 5: 1}
575
+ )
576
+ vindr_df = vindr_df.dropna(subset=["label"])
577
+ split_col = "split" if "split" in vindr_df.columns else None
578
+ if split_col:
579
+ vindr_ext = vindr_df[vindr_df[split_col] == "test"].reset_index(drop=True)
580
+ else:
581
+ _, vindr_ext = train_test_split(
582
+ vindr_df, test_size=0.2, stratify=vindr_df["label"],
583
+ random_state=args.seed
584
+ )
585
+ logger.info("VinDr: %d images | cancer: %.1f%%",
586
+ len(vindr_ext), 100 * vindr_ext["label"].mean())
587
+ else:
588
+ logger.warning("VinDr CSV not found β€” external validation skipped.")
589
+
590
+ # ── Class weight ──────────────────────────────────────────────────────────
591
+ n_pos = int(train_df["label"].sum())
592
+ n_neg = len(train_df) - n_pos
593
+ pos_weight = torch.tensor([n_neg / max(n_pos, 1)], device=device)
594
+ logger.info("Class weight: %.1f (cancer: %.1f%%)",
595
+ pos_weight.item(), 100 * n_pos / len(train_df))
596
+
597
+ # ── Progressive resizing schedule ─────────────────────────────────────────
598
+ # Phase 1: 256Γ—256
599
+ # Phase 2 first third: 256Γ—256
600
+ # Phase 2 second third: 384Γ—384
601
+ # Phase 2 final third: 512Γ—512 (if GPU memory allows) else 384Γ—384
602
+ def get_size(epoch, phase2_epochs, is_phase2):
603
+ if not is_phase2:
604
+ return 256
605
+ third = phase2_epochs // 3
606
+ if epoch <= third:
607
+ return 256
608
+ elif epoch <= 2 * third:
609
+ return 384
610
+ return 512
611
+
612
+ val_tf_256 = make_val_tf(256)
613
+ val_tf_512 = make_val_tf(512)
614
+
615
+ def make_criterion():
616
+ return FocalLoss(
617
+ alpha = args.focal_alpha,
618
+ gamma = args.focal_gamma,
619
+ pos_weight = torch.tensor([1.0, pos_weight.item()], device=device),
620
+ label_smoothing = 0.1,
621
+ )
622
+
623
+ # ── PHASE 1 ───────────────────────────────────────────────────────────────
624
+ if not args.skip_phase1:
625
+ logger.info("\n" + "─" * 70)
626
+ logger.info(" PHASE 1 β€” Head only, 256Γ—256 (%d epochs)", args.phase1_epochs)
627
+ logger.info("─" * 70)
628
+
629
+ model = MultiViewMammogramClassifier(pretrained=True, freeze_backbone=True).to(device)
630
+ criterion = make_criterion()
631
+ optimizer = AdamW(
632
+ [p for p in model.parameters() if p.requires_grad],
633
+ lr=args.phase1_lr, weight_decay=1e-4
634
+ )
635
+ scaler = torch.cuda.amp.GradScaler()
636
+
637
+ train_ds = MultiViewDataset(train_df, rsna_dir, make_train_tf(256), 256)
638
+ val_ds = MultiViewDataset(val_df, rsna_dir, val_tf_256, 256)
639
+ train_loader = DataLoader(train_ds, args.batch_size, True,
640
+ num_workers=args.num_workers, pin_memory=True)
641
+ val_loader = DataLoader(val_ds, args.batch_size, False,
642
+ num_workers=args.num_workers, pin_memory=True)
643
+
644
+ scheduler = OneCycleLR(
645
+ optimizer, max_lr=args.phase1_lr * 3,
646
+ steps_per_epoch=len(train_loader),
647
+ epochs=args.phase1_epochs, pct_start=0.3,
648
+ )
649
+
650
+ best_auc = 0.0
651
+ for epoch in range(1, args.phase1_epochs + 1):
652
+ t0 = time.time()
653
+ tr_loss, tr_sens = run_train_epoch(
654
+ model, train_loader, criterion, optimizer, scheduler, scaler, device
655
+ )
656
+ vl_labels, vl_probs = run_eval_epoch(model, val_loader, device)
657
+ vl_m = compute_metrics(vl_labels, vl_probs)
658
+ logger.info(
659
+ "P1 E%02d/%d | loss=%.4f sens=%.3f | AUC=%.4f sens=%.3f | %.0fs",
660
+ epoch, args.phase1_epochs, tr_loss, tr_sens,
661
+ vl_m["auc"], vl_m["sensitivity"], time.time() - t0,
662
+ )
663
+ if vl_m["auc"] > best_auc:
664
+ best_auc = vl_m["auc"]
665
+ save_checkpoint(model, optimizer, epoch, vl_m,
666
+ output_dir / "mammogram_phase1.pth")
667
+
668
+ logger.info("Phase 1 complete. Best AUC: %.4f", best_auc)
669
+
670
+ # ── PHASE 2 ───────────────────────────────────────────────────────────────
671
+ logger.info("\n" + "─" * 70)
672
+ logger.info(" PHASE 2 β€” Full fine-tuning, progressive resizing (%d epochs)",
673
+ args.phase2_epochs)
674
+ logger.info("─" * 70)
675
+
676
+ model = MultiViewMammogramClassifier(pretrained=False, freeze_backbone=False).to(device)
677
+ phase1_ckpt = output_dir / "mammogram_phase1.pth"
678
+ if phase1_ckpt.exists():
679
+ ckpt = torch.load(phase1_ckpt, map_location=device)
680
+ model.load_state_dict(ckpt["state_dict"])
681
+ logger.info("Loaded Phase 1 (AUC=%.4f)", ckpt["metrics"].get("auc", 0))
682
+
683
+ criterion = make_criterion()
684
+ optimizer = AdamW(model.parameters(), lr=args.phase2_lr, weight_decay=1e-4)
685
+ scaler = torch.cuda.amp.GradScaler()
686
+
687
+ # Build val loaders
688
+ val_ds_256 = MultiViewDataset(val_df, rsna_dir, val_tf_256, 256)
689
+ val_loader = DataLoader(val_ds_256, args.batch_size, False,
690
+ num_workers=args.num_workers, pin_memory=True)
691
+
692
+ vindr_loader = None
693
+ if len(vindr_ext) > 0:
694
+ vindr_ds = VinDrDataset(vindr_ext, vindr_dir, val_tf_256)
695
+ vindr_loader = DataLoader(vindr_ds, args.batch_size, False,
696
+ num_workers=args.num_workers, pin_memory=True)
697
+
698
+ # Compute total steps for scheduler
699
+ train_ds_tmp = MultiViewDataset(train_df, rsna_dir, make_train_tf(256), 256)
700
+ tmp_loader = DataLoader(train_ds_tmp, args.batch_size, True, num_workers=0)
701
+ steps_per_epoch = len(tmp_loader)
702
+ del tmp_loader, train_ds_tmp
703
+
704
+ scheduler = OneCycleLR(
705
+ optimizer, max_lr=args.max_lr,
706
+ steps_per_epoch=steps_per_epoch,
707
+ epochs=args.phase2_epochs, pct_start=0.3,
708
+ )
709
+
710
+ best_auc = 0.0
711
+ best_epoch = 0
712
+ best_thr = 0.5
713
+ log_rows = []
714
+ current_size = 256
715
+ train_loader = DataLoader(
716
+ MultiViewDataset(train_df, rsna_dir, make_train_tf(256), 256),
717
+ args.batch_size, True, num_workers=args.num_workers, pin_memory=True,
718
+ )
719
+
720
+ size_milestones = {
721
+ 1: 256,
722
+ args.phase2_epochs // 3 + 1: 384,
723
+ args.phase2_epochs * 2 // 3 + 1: 512,
724
+ }
725
+
726
+ for epoch in range(1, args.phase2_epochs + 1):
727
+ t0 = time.time()
728
+
729
+ # Progressive resizing
730
+ if epoch in size_milestones:
731
+ new_size = size_milestones[epoch]
732
+ if new_size != current_size:
733
+ current_size = new_size
734
+ logger.info(" β†’ Resizing to %dΓ—%d", current_size, current_size)
735
+ train_loader = DataLoader(
736
+ MultiViewDataset(train_df, rsna_dir,
737
+ make_train_tf(current_size), current_size),
738
+ args.batch_size, True,
739
+ num_workers=args.num_workers, pin_memory=True,
740
+ )
741
+
742
+ tr_loss, tr_sens = run_train_epoch(
743
+ model, train_loader, criterion, optimizer, scheduler, scaler, device
744
+ )
745
+
746
+ rsna_labels, rsna_probs = run_eval_epoch(model, val_loader, device)
747
+ thr = youden_threshold(rsna_labels, rsna_probs)
748
+ rsna_m = compute_metrics(rsna_labels, rsna_probs, thr)
749
+
750
+ vindr_str = ""
751
+ vindr_m = {}
752
+ if vindr_loader:
753
+ vl, vp = run_eval_epoch(model, vindr_loader, device, is_multiview=False)
754
+ vindr_m = compute_metrics(vl, vp, thr)
755
+ vindr_str = f" | VinDr AUC={vindr_m['auc']:.4f} sens={vindr_m['sensitivity']:.3f}"
756
+
757
+ logger.info(
758
+ "E%02d/%d [%3dpx] | loss=%.4f sens=%.3f | RSNA AUC=%.4f sens=%.3f%s | %.0fs",
759
+ epoch, args.phase2_epochs, current_size, tr_loss, tr_sens,
760
+ rsna_m["auc"], rsna_m["sensitivity"], vindr_str, time.time() - t0,
761
+ )
762
+
763
+ log_rows.append({
764
+ "epoch": epoch, "size": current_size,
765
+ "train_loss": tr_loss, "train_sens": tr_sens,
766
+ "rsna_auc": rsna_m["auc"], "rsna_sens": rsna_m["sensitivity"],
767
+ "rsna_spec": rsna_m["specificity"],
768
+ "vindr_auc": vindr_m.get("auc", 0),
769
+ "vindr_sens": vindr_m.get("sensitivity", 0),
770
+ "threshold": thr,
771
+ })
772
+
773
+ if rsna_m["auc"] > best_auc:
774
+ best_auc = rsna_m["auc"]
775
+ best_epoch = epoch
776
+ best_thr = thr
777
+ save_checkpoint(
778
+ model, optimizer, epoch,
779
+ {"rsna": rsna_m, "vindr": vindr_m, "threshold": thr},
780
+ output_dir / "mammogram_weights.pth",
781
+ )
782
+ logger.info(" βœ“ Best checkpoint (AUC=%.4f)", best_auc)
783
+
784
+ # ── Final evaluation ──────────────────────────────────────────────────────
785
+ logger.info("\n" + "=" * 70)
786
+ logger.info(" FINAL EVALUATION (epoch %d)", best_epoch)
787
+ logger.info("=" * 70)
788
+
789
+ ckpt = torch.load(output_dir / "mammogram_weights.pth", map_location=device)
790
+ model.load_state_dict(ckpt["state_dict"])
791
+
792
+ rsna_l, rsna_p = run_eval_epoch(model, val_loader, device)
793
+ rsna_f = compute_metrics(rsna_l, rsna_p, best_thr)
794
+ rsna_ci = bootstrap_auc_ci(rsna_l, rsna_p)
795
+
796
+ logger.info("RSNA (internal): AUC=%.4f (95%% CI: %.4f–%.4f) sens=%.1f%% spec=%.1f%%",
797
+ rsna_f["auc"], *rsna_ci,
798
+ rsna_f["sensitivity"] * 100, rsna_f["specificity"] * 100)
799
+
800
+ vindr_f = {}
801
+ vindr_ci = (0, 0)
802
+ if vindr_loader:
803
+ vl, vp = run_eval_epoch(model, vindr_loader, device, is_multiview=False)
804
+ vindr_f = compute_metrics(vl, vp, best_thr)
805
+ vindr_ci = bootstrap_auc_ci(vl, vp)
806
+ gap = rsna_f["auc"] - vindr_f["auc"]
807
+ logger.info("VinDr (external): AUC=%.4f (95%% CI: %.4f–%.4f) sens=%.1f%% spec=%.1f%%",
808
+ vindr_f["auc"], *vindr_ci,
809
+ vindr_f["sensitivity"] * 100, vindr_f["specificity"] * 100)
810
+ logger.info("Generalisation gap: %.4f (%s)",
811
+ gap,
812
+ "βœ“ Excellent" if gap < 0.05 else "⚠ Moderate" if gap < 0.10 else "βœ— Large")
813
+
814
+ # ── Save results ──────────────────────────────────────────────────────────
815
+ with open(output_dir / "mammogram_training_log.csv", "w", newline="") as f:
816
+ w = csv.DictWriter(f, fieldnames=log_rows[0].keys())
817
+ w.writeheader()
818
+ w.writerows(log_rows)
819
+
820
+ results = {
821
+ "best_epoch": best_epoch,
822
+ "threshold": best_thr,
823
+ "rsna_internal": {**rsna_f, "auc_ci": rsna_ci},
824
+ "vindr_external": {**vindr_f, "auc_ci": vindr_ci} if vindr_f else {},
825
+ "generalisation_gap": round(rsna_f["auc"] - vindr_f.get("auc", rsna_f["auc"]), 4),
826
+ }
827
+ with open(output_dir / "mammogram_results.json", "w") as f:
828
+ json.dump(results, f, indent=2)
829
+
830
+ logger.info("\nSaved to %s:", output_dir)
831
+ logger.info(" mammogram_weights.pth")
832
+ logger.info(" mammogram_training_log.csv")
833
+ logger.info(" mammogram_results.json")
834
+ logger.info("=" * 70)
835
+
836
+
837
+ if __name__ == "__main__":
838
+ main()
utils/mammogram_preprocessing.py ADDED
@@ -0,0 +1,280 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ utils/mammogram_preprocessing.py
3
+ ──────────────────────────────────
4
+ Preprocessing pipeline for full-field digital mammography (FFDM).
5
+
6
+ Handles both DICOM files and standard image formats (PNG, JPG).
7
+ Applies mammogram-specific preprocessing:
8
+ - VOI LUT windowing for DICOM files
9
+ - Breast region normalisation
10
+ - Grayscale to 3-channel RGB conversion
11
+ - Mammogram-appropriate augmentations (no stain jitter)
12
+
13
+ Install
14
+ ───────
15
+ pip install pydicom pylibjpeg python-gdcm
16
+ """
17
+
18
+ from __future__ import annotations
19
+
20
+ from pathlib import Path
21
+ from typing import Optional, Union
22
+
23
+ import numpy as np
24
+ from PIL import Image
25
+ from torchvision import transforms
26
+
27
+ # DICOM support β€” optional import so the rest of the codebase doesn't break
28
+ # if pydicom is not installed
29
+ try:
30
+ import pydicom
31
+ from pydicom.pixel_data_handlers.util import apply_voi_lut
32
+ PYDICOM_AVAILABLE = True
33
+ except ImportError:
34
+ PYDICOM_AVAILABLE = False
35
+
36
+ # ── Constants ──────────────────────────────────────────────────────────────────
37
+ MAMMOGRAM_SIZE = 512 # EfficientNet-B4 input resolution
38
+ IMAGENET_MEAN = [0.485, 0.456, 0.406]
39
+ IMAGENET_STD = [0.229, 0.224, 0.225]
40
+
41
+
42
+ # ── DICOM loader ───────────────────────────────────────────────────────────────
43
+
44
+ def load_dicom(path: Union[str, Path]) -> Image.Image:
45
+ """
46
+ Load a DICOM mammogram file and convert to an 8-bit RGB PIL Image.
47
+
48
+ Applies VOI LUT windowing if available in the DICOM metadata,
49
+ otherwise falls back to min-max normalisation.
50
+
51
+ MONOCHROME1 images (where high pixel = dark) are inverted so
52
+ the tissue appears bright on a dark background, matching the
53
+ visual convention used during model training.
54
+
55
+ Parameters
56
+ ----------
57
+ path : str | Path
58
+ Path to a .dcm DICOM file.
59
+
60
+ Returns
61
+ -------
62
+ PIL.Image.Image β€” RGB image ready for preprocessing.
63
+
64
+ Raises
65
+ ------
66
+ ImportError β€” if pydicom is not installed.
67
+ FileNotFoundError β€” if the file does not exist.
68
+ """
69
+ if not PYDICOM_AVAILABLE:
70
+ raise ImportError(
71
+ "pydicom not installed. Run: pip install pydicom pylibjpeg python-gdcm"
72
+ )
73
+
74
+ path = Path(path)
75
+ if not path.exists():
76
+ raise FileNotFoundError(f"DICOM file not found: {path}")
77
+
78
+ dcm = pydicom.dcmread(str(path))
79
+
80
+ # Apply VOI LUT windowing (converts to display-ready values)
81
+ try:
82
+ pixel_array = apply_voi_lut(dcm.pixel_array, dcm)
83
+ except Exception:
84
+ pixel_array = dcm.pixel_array.astype(np.float32)
85
+
86
+ pixel_array = pixel_array.astype(np.float32)
87
+
88
+ # Normalise to [0, 255]
89
+ p_min, p_max = pixel_array.min(), pixel_array.max()
90
+ if p_max > p_min:
91
+ pixel_array = (pixel_array - p_min) / (p_max - p_min) * 255.0
92
+ else:
93
+ pixel_array = np.zeros_like(pixel_array)
94
+
95
+ pixel_array = pixel_array.astype(np.uint8)
96
+
97
+ # MONOCHROME1: invert so tissue is bright
98
+ if hasattr(dcm, "PhotometricInterpretation"):
99
+ if dcm.PhotometricInterpretation == "MONOCHROME1":
100
+ pixel_array = 255 - pixel_array
101
+
102
+ # Convert grayscale β†’ RGB (EfficientNet expects 3 channels)
103
+ if pixel_array.ndim == 2:
104
+ rgb = np.stack([pixel_array, pixel_array, pixel_array], axis=-1)
105
+ else:
106
+ rgb = pixel_array
107
+
108
+ return Image.fromarray(rgb, mode="RGB")
109
+
110
+
111
+ def load_mammogram(path: Union[str, Path]) -> Image.Image:
112
+ """
113
+ Load a mammogram from either DICOM or standard image format.
114
+
115
+ Automatically detects format by file extension.
116
+
117
+ Parameters
118
+ ----------
119
+ path : str | Path
120
+ Path to DICOM (.dcm) or image file (.png, .jpg, .tiff).
121
+
122
+ Returns
123
+ -------
124
+ PIL.Image.Image β€” RGB image.
125
+ """
126
+ path = Path(path)
127
+ if path.suffix.lower() in {".dcm", ".dicom"}:
128
+ return load_dicom(path)
129
+
130
+ # Standard image format
131
+ img = Image.open(path).convert("RGB")
132
+
133
+ # If grayscale was saved as single-channel PNG, already converted above.
134
+ # But if it's a true grayscale mammogram saved as PNG:
135
+ if img.mode == "L":
136
+ arr = np.array(img)
137
+ rgb = np.stack([arr, arr, arr], axis=-1)
138
+ return Image.fromarray(rgb, mode="RGB")
139
+
140
+ return img
141
+
142
+
143
+ # ── Mammogram-specific augmentation transforms ──────────────────────────────────
144
+
145
+ class BreastRegionEnhancer:
146
+ """
147
+ Enhances breast tissue contrast using adaptive histogram equalisation.
148
+
149
+ Applied per-channel to improve visibility of masses and calcifications
150
+ without affecting the overall image structure.
151
+
152
+ Parameters
153
+ ----------
154
+ clip_limit : float
155
+ CLAHE clip limit. Higher = more aggressive enhancement.
156
+ p : float
157
+ Probability of applying this transform.
158
+ """
159
+
160
+ def __init__(self, clip_limit: float = 2.0, p: float = 0.5) -> None:
161
+ self.clip_limit = clip_limit
162
+ self.p = p
163
+
164
+ def __call__(self, img: Image.Image) -> Image.Image:
165
+ if np.random.random() > self.p:
166
+ return img
167
+
168
+ try:
169
+ import cv2
170
+ arr = np.array(img)
171
+ clahe = cv2.createCLAHE(
172
+ clipLimit = self.clip_limit,
173
+ tileGridSize = (8, 8),
174
+ )
175
+ # Apply CLAHE to each channel
176
+ enhanced = np.stack(
177
+ [clahe.apply(arr[:, :, c]) for c in range(arr.shape[2])],
178
+ axis=-1,
179
+ )
180
+ return Image.fromarray(enhanced.astype(np.uint8), mode="RGB")
181
+ except ImportError:
182
+ return img # OpenCV not available β€” skip silently
183
+
184
+
185
+ class RandomElasticDeformation:
186
+ """
187
+ Random elastic deformation β€” simulates tissue compression variation
188
+ from different mammography unit pressures.
189
+
190
+ Parameters
191
+ ----------
192
+ alpha : float
193
+ Strength of deformation.
194
+ sigma : float
195
+ Smoothness of deformation field.
196
+ p : float
197
+ Probability of applying.
198
+ """
199
+
200
+ def __init__(
201
+ self,
202
+ alpha: float = 34.0,
203
+ sigma: float = 4.0,
204
+ p: float = 0.3,
205
+ ) -> None:
206
+ self.alpha = alpha
207
+ self.sigma = sigma
208
+ self.p = p
209
+
210
+ def __call__(self, img: Image.Image) -> Image.Image:
211
+ if np.random.random() > self.p:
212
+ return img
213
+
214
+ try:
215
+ from scipy.ndimage import gaussian_filter, map_coordinates
216
+
217
+ arr = np.array(img, dtype=np.float32)
218
+ h, w = arr.shape[:2]
219
+ dx = gaussian_filter(
220
+ (np.random.rand(h, w) * 2 - 1) * self.alpha, self.sigma
221
+ )
222
+ dy = gaussian_filter(
223
+ (np.random.rand(h, w) * 2 - 1) * self.alpha, self.sigma
224
+ )
225
+ x, y = np.meshgrid(np.arange(w), np.arange(h))
226
+ coords = [
227
+ np.clip(y + dy, 0, h - 1).ravel(),
228
+ np.clip(x + dx, 0, w - 1).ravel(),
229
+ ]
230
+
231
+ result = np.stack([
232
+ map_coordinates(arr[:, :, c], coords, order=1).reshape(h, w)
233
+ for c in range(arr.shape[2])
234
+ ], axis=-1)
235
+
236
+ return Image.fromarray(result.clip(0, 255).astype(np.uint8), mode="RGB")
237
+ except ImportError:
238
+ return img # scipy not available β€” skip
239
+
240
+
241
+ def build_mammogram_train_transform() -> transforms.Compose:
242
+ """
243
+ Training augmentation pipeline for mammograms.
244
+
245
+ Mammogram-appropriate augmentations β€” NO stain jitter (mammograms
246
+ are X-ray images, not H&E-stained tissue). Augmentations simulate:
247
+ - Different patient positioning (flips, rotation)
248
+ - Tissue compression variation (elastic deformation)
249
+ - Scanner variation (brightness/contrast)
250
+ - Tissue contrast differences (CLAHE)
251
+ """
252
+ return transforms.Compose([
253
+ BreastRegionEnhancer(clip_limit=2.0, p=0.4),
254
+ RandomElasticDeformation(alpha=34.0, sigma=4.0, p=0.3),
255
+ transforms.Resize((MAMMOGRAM_SIZE, MAMMOGRAM_SIZE)),
256
+ transforms.RandomHorizontalFlip(p=0.5),
257
+ transforms.RandomVerticalFlip(p=0.2),
258
+ transforms.RandomRotation(degrees=10),
259
+ transforms.ColorJitter(
260
+ brightness = 0.15,
261
+ contrast = 0.15,
262
+ ),
263
+ transforms.RandomAffine(
264
+ degrees = 0,
265
+ translate = (0.05, 0.05),
266
+ scale = (0.95, 1.05),
267
+ ),
268
+ transforms.ToTensor(),
269
+ transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
270
+ ])
271
+
272
+
273
+ def build_mammogram_inference_transform() -> transforms.Compose:
274
+ """
275
+ Inference transform β€” resize and normalise only. No augmentation."""
276
+ return transforms.Compose([
277
+ transforms.Resize((MAMMOGRAM_SIZE, MAMMOGRAM_SIZE)),
278
+ transforms.ToTensor(),
279
+ transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
280
+ ])
utils/preprocessing.py ADDED
@@ -0,0 +1,211 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ utils/preprocessing.py
3
+ ───────────────────────
4
+ Image preprocessing pipeline that converts a raw histopathology image
5
+ (file path, PIL Image, or numpy array) into a normalised tensor of
6
+ shape (1, 3, 224, 224) ready for model inference.
7
+
8
+ Normalization follows ImageNet standards as required by the spec:
9
+ Mean : [0.485, 0.456, 0.406]
10
+ Std : [0.229, 0.224, 0.225]
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ from pathlib import Path
16
+ from typing import Union
17
+
18
+ import numpy as np
19
+ import torch
20
+ from PIL import Image
21
+ from torchvision import transforms
22
+
23
+ # ── ImageNet normalization constants ────────────────────────────────────────
24
+ IMAGENET_MEAN = [0.485, 0.456, 0.406]
25
+ IMAGENET_STD = [0.229, 0.224, 0.225]
26
+
27
+ # ── Target tensor shape ─────────────────────────────────────────────────────
28
+ TARGET_SIZE = (224, 224)
29
+
30
+
31
+ # ── Transform pipeline ───────────────────────────────────────────────────────
32
+ def build_inference_transform() -> transforms.Compose:
33
+ """
34
+ Returns the deterministic inference transform pipeline.
35
+
36
+ Steps
37
+ ─────
38
+ 1. Resize shortest edge to 256 px (preserves aspect ratio).
39
+ 2. Centre-crop to 224 Γ— 224.
40
+ 3. Convert PIL image to float32 tensor in [0, 1].
41
+ 4. Normalize with ImageNet mean / std.
42
+ """
43
+ return transforms.Compose([
44
+ transforms.Resize(256, interpolation=transforms.InterpolationMode.BICUBIC),
45
+ transforms.CenterCrop(TARGET_SIZE),
46
+ transforms.ToTensor(), # β†’ [0, 1]
47
+ transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
48
+ ])
49
+
50
+
51
+
52
+
53
+ # ── StainJitter β€” Fix 1 ──────────────────────────────────────────────────────
54
+ class StainJitter:
55
+ """
56
+ Randomly perturb H&E stain concentrations in HED colour space.
57
+
58
+ Why this works
59
+ ──────────────
60
+ H&E-stained slides vary in colour between labs due to differences in
61
+ staining batches, fixation protocols, and scanner calibrations.
62
+ Standard RGB colour jitter doesn't model this β€” it shifts all three
63
+ channels independently. StainJitter works in HED space (Haematoxylin,
64
+ Eosin, DAB), which directly corresponds to the actual stains in the tissue.
65
+ Perturbing HED channels simulates real-world staining variation without
66
+ needing a reference image or external library.
67
+
68
+ Implementation
69
+ ──────────────
70
+ Uses the Ruifrok & Johnston HED deconvolution matrix to decompose
71
+ RGB into stain concentrations, perturbs each channel with a random
72
+ scale (alpha) and shift (beta), then reconstructs the RGB image.
73
+ Pure NumPy β€” no external dependencies beyond what is already installed.
74
+
75
+ Parameters
76
+ ----------
77
+ strength : float
78
+ Controls the magnitude of perturbation.
79
+ 0.05 = Β±5% scale variation + Β±5% shift variation per channel.
80
+ Typical values: 0.03 (subtle) to 0.10 (aggressive).
81
+ p : float
82
+ Probability of applying the transform. Default 0.5.
83
+ """
84
+
85
+ # Ruifrok & Johnston HED deconvolution matrix
86
+ # Rows = [Haematoxylin, Eosin, DAB] stain absorption vectors
87
+ HED = np.array([
88
+ [0.6500286, 0.7044536, 0.2860126],
89
+ [0.7044522, 0.4956977, 0.5079795],
90
+ [0.2860126, 0.5079795, 0.8128560],
91
+ ], dtype=np.float64)
92
+
93
+ # Pre-compute inverse once at class level
94
+ HED_INV = np.linalg.inv(HED)
95
+
96
+ def __init__(self, strength: float = 0.05, p: float = 0.5) -> None:
97
+ self.strength = strength
98
+ self.p = p
99
+
100
+ def __call__(self, img: "Image.Image") -> "Image.Image":
101
+ if np.random.random() > self.p:
102
+ return img
103
+
104
+ # PIL β†’ float64 numpy in [0, 1]
105
+ rgb = np.array(img, dtype=np.float64) / 255.0
106
+
107
+ # Convert to optical density β€” Beer-Lambert law
108
+ # Clamp to avoid log(0)
109
+ od = -np.log(np.clip(rgb, 1e-6, 1.0))
110
+
111
+ # Decompose into HED stain concentrations
112
+ # od = concentrations @ HED β†’ concentrations = od @ HED_INV
113
+ hed = od @ self.HED_INV # (H, W, 3) HED concentrations
114
+
115
+ # Perturb each stain channel independently
116
+ alpha = np.random.uniform(
117
+ 1.0 - self.strength,
118
+ 1.0 + self.strength,
119
+ size=(1, 1, 3),
120
+ )
121
+ beta = np.random.uniform(
122
+ -self.strength,
123
+ +self.strength,
124
+ size=(1, 1, 3),
125
+ )
126
+ hed_perturbed = hed * alpha + beta
127
+
128
+ # Reconstruct optical density then RGB
129
+ od_reconstructed = hed_perturbed @ self.HED
130
+ rgb_out = np.exp(-od_reconstructed)
131
+ rgb_out = np.clip(rgb_out, 0.0, 1.0)
132
+
133
+ return Image.fromarray((rgb_out * 255).astype(np.uint8), mode="RGB")
134
+
135
+ def build_training_transform() -> transforms.Compose:
136
+ """
137
+ Augmentation pipeline for fine-tuning.
138
+ Included for completeness; inference always uses build_inference_transform().
139
+ """
140
+ return transforms.Compose([
141
+ StainJitter(strength=0.05, p=0.5), # Fix 1: H&E stain augmentation
142
+ transforms.RandomResizedCrop(TARGET_SIZE, scale=(0.8, 1.0)),
143
+ transforms.RandomHorizontalFlip(),
144
+ transforms.RandomVerticalFlip(),
145
+ transforms.ColorJitter(brightness=0.2, contrast=0.2,
146
+ saturation=0.1),
147
+ transforms.RandomRotation(degrees=15),
148
+ transforms.ToTensor(),
149
+ transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
150
+ ])
151
+
152
+
153
+ # ── Preprocessing entry point ────────────────────────────────────────────────
154
+ class ImagePreprocessor:
155
+ """
156
+ Accepts multiple input types and returns a (1, 3, 224, 224) tensor.
157
+
158
+ Supported inputs
159
+ ────────────────
160
+ - str / pathlib.Path : local file path to PNG / JPG / TIFF
161
+ - PIL.Image.Image : already-loaded PIL image
162
+ - np.ndarray : HxWx3 uint8 or float32 array
163
+ - torch.Tensor : CxHxW or 1xCxHxW (skips PIL stage)
164
+ """
165
+
166
+ def __init__(self) -> None:
167
+ self._transform = build_inference_transform()
168
+
169
+ def __call__(
170
+ self,
171
+ image: Union[str, Path, "Image.Image", np.ndarray, torch.Tensor],
172
+ ) -> torch.Tensor:
173
+ """
174
+ Parameters
175
+ ----------
176
+ image : see supported inputs above
177
+
178
+ Returns
179
+ -------
180
+ torch.Tensor
181
+ Shape (1, 3, 224, 224), dtype float32, ImageNet-normalised.
182
+ """
183
+ pil_image = self._to_pil(image)
184
+ tensor = self._transform(pil_image) # (3, 224, 224)
185
+ return tensor.unsqueeze(0) # (1, 3, 224, 224)
186
+
187
+ # ── Type dispatch helpers ────────────────────────────────────────────────
188
+ @staticmethod
189
+ def _to_pil(image) -> "Image.Image":
190
+ if isinstance(image, (str, Path)):
191
+ return Image.open(image).convert("RGB")
192
+
193
+ if isinstance(image, Image.Image):
194
+ return image.convert("RGB")
195
+
196
+ if isinstance(image, np.ndarray):
197
+ if image.dtype != np.uint8:
198
+ image = (np.clip(image, 0, 1) * 255).astype(np.uint8)
199
+ if image.ndim == 2:
200
+ image = np.stack([image] * 3, axis=-1) # grayscale β†’ RGB
201
+ return Image.fromarray(image, mode="RGB")
202
+
203
+ if isinstance(image, torch.Tensor):
204
+ t = image.squeeze(0) if image.ndim == 4 else image
205
+ arr = (t.permute(1, 2, 0).cpu().numpy() * 255).astype(np.uint8)
206
+ return Image.fromarray(arr, mode="RGB")
207
+
208
+ raise TypeError(
209
+ f"Unsupported image type: {type(image)}. "
210
+ "Expected str, Path, PIL.Image, np.ndarray, or torch.Tensor."
211
+ )