mstepien commited on
Commit
a63cedf
·
0 Parent(s):

Dermatolog AI Scan

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
.devcontainer/devcontainer.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "Dermatolog AI Scanner Dev",
3
+ "build": {
4
+ "context": "..",
5
+ "dockerfile": "../Dockerfile"
6
+ },
7
+ "customizations": {
8
+ "vscode": {
9
+ "extensions": [
10
+ "ms-python.python",
11
+ "ms-python.black-formatter",
12
+ "charliermarsh.ruff",
13
+ "tamasfe.even-better-toml"
14
+ ]
15
+ }
16
+ },
17
+ "forwardPorts": [
18
+ 8000
19
+ ],
20
+ "postCreateCommand": "pip install -r requirements-dev.txt && playwright install --with-deps chromium && npm install",
21
+ "runArgs": [
22
+ "--env-file",
23
+ ".env"
24
+ ]
25
+ }
.dockerignore ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ __pycache__
2
+ *.pyc
3
+ *.pyo
4
+ *.pyd
5
+ .Python
6
+ env/
7
+ venv/
8
+ pip-log.txt
9
+ pip-delete-this-directory.txt
10
+ .tox/
11
+ .coverage
12
+ .coverage.*
13
+ .cache
14
+ nosetests.xml
15
+ coverage.xml
16
+ *.cover
17
+ *.log
18
+ .git
19
+ .mypy_cache
20
+ .pytest_cache
21
+ .hypothesize
22
+
23
+ # Data and Cache
24
+ tmp/
25
+ db/
26
+ cache/
27
+ model_cache/
28
+
29
+ # Local Config
30
+ .env
31
+ .DS_Store
.env ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ # GCP Project Configuration
2
+ PROJECT_ID=your-project-id
3
+ LOCATION=europe-central2
4
+
5
+
6
+
.gitignore ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[codz]
4
+ *$py.class
5
+ server.log
6
+ debug*.log
7
+ image_stats.duckdb
8
+ venv/
9
+ db/
10
+ tmp/
11
+ coverage/coverage-final.json
12
+ node_modules/*
13
+ coverage/*
14
+ app/static/js/client/git_push.sh
15
+ app/static/js/client/mocha.opts
16
+ *.DS_Store
17
+ gcp_key.json
18
+ verify*.py
19
+ test_*.log
20
+ *.out
21
+ .env
22
+ *.db
23
+ yolov8n.pt
.nvmrc ADDED
@@ -0,0 +1 @@
 
 
1
+ 18
DEPLOY.md ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 📦 Deployment Guide
2
+
3
+ This application is fully containerized and can be deployed to Google Cloud Run, AWS, or any Kubernetes cluster.
4
+
5
+ ## Minimum Requirements
6
+
7
+ - **RAM**: 4 GB (8 GB Recommended for MedSigLIP model)
8
+ - **CPU**: 2 vCPU
9
+ - **Dependencies**: Docker (for building the image)
10
+
11
+ ---
12
+
13
+ ## 🚀 Google Cloud Run
14
+
15
+ We provide a helper script to deploy with the correct hardware configuration.
16
+
17
+ 1. **Authenticate**:
18
+ ```bash
19
+ gcloud auth login
20
+ gcloud config set project YOUR_PROJECT_ID
21
+ ```
22
+
23
+ 2. **Export HF_TOKEN (Crucial)**:
24
+ For the build to succeed (downloading gated model), you must export your token:
25
+ ```bash
26
+ export HF_TOKEN=your_hf_token
27
+ ```
28
+
29
+ 3. **Run Deployment Script**:
30
+ ```bash
31
+ chmod +x bin/deploy.sh
32
+ ./bin/deploy.sh
33
+ ```
34
+
35
+ This script will:
36
+ - Build the container image.
37
+ - Deploy to Cloud Run with **8GB RAM** and **2 vCPUs**.
38
+ - Configure the fallback to public models if no gated token is provided.
39
+
40
+ 4. **Access**:
41
+ The script will output the public URL of your application.
42
+
43
+ ### 🔧 Cloud Build Configuration (`cloudbuild.yaml`)
44
+
45
+ The project includes a `cloudbuild.yaml` file, which is used by Google Cloud Build to execute the container build process.
46
+
47
+ **Why is it needed?**
48
+ The standard `gcloud builds submit` command does not support passing build arguments (like `HF_TOKEN`) directly to the Dockerfile easily. The `cloudbuild.yaml` file explicitly defines the build steps to include the `--build-arg` flag, ensuring the gated MedSigLIP model can be downloaded securely during the build.
49
+
50
+ **Manual Usage:**
51
+ If you need to trigger a build manually without `bin/deploy.sh`:
52
+ ```bash
53
+ gcloud builds submit --config cloudbuild.yaml \
54
+ --substitutions=_HF_TOKEN="$HF_TOKEN",_SERVICE_NAME="dermatolog-ai-scan" .
55
+ ```
56
+
57
+
58
+ ## AWS (Amazon Web Services)
59
+
60
+ You can deploy using **AWS App Runner** (easiest) or **Amazon ECS**.
61
+
62
+ 1. **Build and Push Image**:
63
+ Creating an ECR repository and pushing your image:
64
+ ```bash
65
+ aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
66
+
67
+ docker build -t dermatolog-ai-scan .
68
+ docker tag dermatolog-ai-scan:latest YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/dermatolog-ai-scan:latest
69
+ docker push YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/dermatolog-ai-scan:latest
70
+ ```
71
+
72
+ 2. **Deploy via App Runner**:
73
+ - Select **Container Registry** in App Runner.
74
+ - Choose the pushed image.
75
+ - **Configuration**:
76
+ - **CPU**: 2 vCPU
77
+ - **Memory**: 4 GB (Minimum) or higher.
78
+ - **Port**: 8000
79
+ - **Environment Variables**: Add `HF_TOKEN` if you have one.
80
+
81
+ ---
82
+
83
+ ## ☸️ Kubernetes (K8s)
84
+
85
+ Deploy to any Kubernetes formatted cluster (EKS, GKE, K3s, Minikube).
86
+
87
+ **1. Create Deployment (`k8s-deployment.yaml`)**:
88
+ ```yaml
89
+ apiVersion: apps/v1
90
+ kind: Deployment
91
+ metadata:
92
+ name: dermatolog-ai
93
+ spec:
94
+ replicas: 1
95
+ selector:
96
+ matchLabels:
97
+ app: dermatolog-ai
98
+ template:
99
+ metadata:
100
+ labels:
101
+ app: dermatolog-ai
102
+ spec:
103
+ containers:
104
+ - name: dermatolog-ai
105
+ image: your-registry/dermatolog-ai-scan:latest
106
+ resources:
107
+ requests:
108
+ memory: "4Gi"
109
+ cpu: "1000m"
110
+ limits:
111
+ memory: "8Gi"
112
+ cpu: "2000m"
113
+ ports:
114
+ - containerPort: 8000
115
+ env:
116
+ # Optional: Add HF_TOKEN secret if using gated models
117
+ # - name: HF_TOKEN
118
+ # valueFrom:
119
+ # secretKeyRef:
120
+ # name: hf-secret
121
+ # key: token
122
+ ```
123
+
124
+ **2. Expose Service (`k8s-service.yaml`)**:
125
+ ```yaml
126
+ apiVersion: v1
127
+ kind: Service
128
+ metadata:
129
+ name: dermatolog-ai-service
130
+ spec:
131
+ type: LoadBalancer
132
+ selector:
133
+ app: dermatolog-ai
134
+ ports:
135
+ - protocol: TCP
136
+ port: 80
137
+ targetPort: 8000
138
+ ```
139
+
140
+ **3. Apply Configuration**:
141
+ ```bash
142
+ kubectl apply -f k8s-deployment.yaml
143
+ kubectl apply -f k8s-service.yaml
144
+ ```
Dockerfile ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.11-slim
2
+
3
+ WORKDIR /app
4
+
5
+ # Install system dependencies
6
+ RUN apt-get update && apt-get install -y \
7
+ curl \
8
+ wget \
9
+ build-essential \
10
+ libgl1 \
11
+ libglib2.0-0 \
12
+ && curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
13
+ && apt-get install -y nodejs \
14
+ && rm -rf /var/lib/apt/lists/*
15
+
16
+ # Install python dependencies
17
+ COPY requirements.txt .
18
+ RUN pip install --no-cache-dir -r requirements.txt
19
+
20
+ # Pre-download the model to bake it into the image
21
+ # This prevents downloading 4GB+ on every container start
22
+ ARG HF_TOKEN
23
+ ENV HF_TOKEN=${HF_TOKEN}
24
+
25
+ RUN python -c "from transformers import AutoProcessor, AutoModel; \
26
+ import os; \
27
+ token = os.environ.get('HF_TOKEN'); \
28
+ print(f'Downloading MedSigLIP model with token present: {bool(token)}...'); \
29
+ AutoProcessor.from_pretrained('google/medsiglip-448', token=token); \
30
+ AutoModel.from_pretrained('google/medsiglip-448', token=token)"
31
+
32
+ # Pre-download YOLO model
33
+ RUN python -c "from ultralytics import YOLO; YOLO('yolov8n.pt')"
34
+
35
+ # Copy application code
36
+ COPY . .
37
+
38
+ # Expose port (Cloud Run defaults to 8080, providing a fallback)
39
+ ENV PORT=8080
40
+ EXPOSE $PORT
41
+
42
+ # Command to run (Using Shell form so it evaluates $PORT)
43
+ CMD uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8080}
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Marcin Stepien
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Dermatolog AI Scan
2
+
3
+ A privacy-first, free, and easy-to-use dermatology scan app powered by latest AI models.
4
+
5
+ ## Features
6
+
7
+ - **Local Models**: Direct interface with MedSigLIP model locally or on Cloud Run.
8
+ - **Lesion Detection**: Uses **YOLOv8-Nano** to automatically identify and localise skin lesions for optimized preprocessing.
9
+ - **Session-based Photo Management**:
10
+ - **Local-Only Storage**: Images are processed and stored entirely within your browser's memory using DataURLs. No image files are ever written to the server's disk, ensuring maximum patient privacy.
11
+ - **Drag & Drop Upload**: Upload multiple images easily.
12
+ - **Clipboard Paste Support**: Paste images directly from your clipboard (Ctrl+V) to preview them instantly.
13
+ - **Smart Timeline**: Photos are automatically grouped into "Virtual Directories" based on their creation date (extracted from EXIF).
14
+ - **Privacy**: All data is scoped to your browser session.
15
+ - **Zero-Shot Dermatology Analysis**:
16
+ - Uses **Google Health's MedSigLIP** (`google/medsiglip-448`) model for localized analysis.
17
+ - Classifies images against a comprehensive set of **25+ dermatological conditions** relevant to EU medical practices.
18
+ - **Rationale**: The label set focuses on high-mortality cancers (Melanoma), high-prevalence conditions (Eczema, Acne), and common differential diagnoses to aid in effective triage.
19
+
20
+ ### 📊 Confidence & Interpretation Logic
21
+
22
+ The application uses specialized logic to convert raw model scores into clinical insights:
23
+
24
+ - **Cancerous Tumor Consolidation**: If the top-ranked results are malignant tumor diseases
25
+ (
26
+ Melanoma,
27
+ Basal Cell Carcinoma,
28
+ Squamous Cell Carcinoma,
29
+ Bowen's Disease
30
+ )
31
+ the confidence margin is calculated as the **difference between the sum of these top tumor scores and the first non-tumor result**. This ensures high confidence is reported when the AI is certain of malignancy, even if it is debating the specific tumor subtype.
32
+ - **Predictive Entropy**: The system calculates Shannon Entropy across all predictions. If entropy is high (e.g., above 2.0 bits), the result is flagged as unreliable regardless of the top score.
33
+ - **Interpretation Margin**: For mixed cases (Tumor vs. Non-Tumor), if the margin is below the configurable threshold (default 5%), the application flags the result as "Not clear" to prompt manual review.
34
+
35
+ ### 🩺 Supported Dermatological Conditions
36
+
37
+ The system is tuned to detect the following conditions based on EU referral guidelines and prevalence statistics:
38
+
39
+ | Category | Conditions | Rationale |
40
+ | :--- | :--- | :--- |
41
+ | **Malignant / Pre-malignant** | Melanoma, Basal Cell Carcinoma (BCC), Squamous Cell Carcinoma (SCC), Actinic Keratosis, Bowen's Disease, Dysplastic Nevus | Priority for early detection due to mortality risk (Melanoma) or high prevalence impacting healthcare resources (BCC/SCC). |
42
+ | **Inflammatory** | Psoriasis, Atopic Dermatitis (Eczema), Acne Vulgaris, Rosacea, Urticaria, Lichen Planus, Hidradenitis Suppurativa | Represents the highest burden of disease on quality of life in the EU population. |
43
+ | **Infectious** | Fungal Infections (Tinea), Herpes Zoster (Shingles), Impetigo, Warts, Molluscum Contagiosum | Frequent reasons for primary care visits; contagious nature requires accurate identification. |
44
+ | **Benign / Differential** | Melanocytic Nevus, Seborrheic Keratosis, Dermatofibroma, Haemangioma, Epidermoid Cyst, Lipoma | Crucial for distinguishing from malignant lesions to reduce unnecessary anxiety and referrals. |
45
+ | **Other** | Vitiligo, Alopecia Areata, Melasma | Common pigmentary and hair disorders affecting psychological well-being. |
46
+
47
+ ## 🔒 Privacy & Security
48
+
49
+ Dermatolog AI Scan is built with a **Privacy-First** architecture:
50
+
51
+ 1. **Browser-Side Image Handling**: When you select an image, it is read by the `FileReader` API and converted to a Base64 DataURL.
52
+ 2. **No Server-Side Persistence**: The backend receives the image data only for the duration of the analysis request. It process the image in-memory and returns the results. No temporary or permanent image files are created on the server's filesystem.
53
+ 3. **Local Memory State**: Image data is pinned to the JavaScript state of your current browser tab. Refreshing the page or closing the tab clears the local image memory.
54
+ 4. **Session Isolation**: Each user is assigned a unique, random session ID to isolate their requests and analysis cache.
55
+
56
+
57
+ ## 🚀 Getting Started
58
+
59
+ ### Prerequisites
60
+
61
+ - **Docker** and **Docker Compose** installed.
62
+ - **VS Code** with the **Dev Containers** extension.
63
+ - **Node.js** (v18+) and **npm** (for frontend tests).
64
+
65
+ ### 🛠️ Development Setup
66
+
67
+ The project is designed to be developed inside a **Dev Container**. This ensures a consistent environment with all dependencies pre-installed.
68
+
69
+ 1. **Clone the Repository**:
70
+ ```bash
71
+ git clone <repository-url>
72
+ cd dermatolog-ai-scan
73
+ ```
74
+
75
+ 3. **HuggingFace Configuration**:
76
+ Access to the MedSigLIP model is gated. You must provide a token in your `.env` file to download/load the model.
77
+
78
+
79
+ 4. **Environment Variables (`.env`)**:
80
+
81
+ Create a `.env` file in the root directory to store configuration variables. This file is automatically loaded by:
82
+ - **Docker Compose**: Used to populate `environment:` variables in `docker-compose.yml`.
83
+ - **Development Container**: To set workspace environment variables.
84
+ - **Deployment Script**: `bin/deploy.sh` reads `PROJECT_ID` from this file.
85
+
86
+
87
+ **Template `.env`:**
88
+ ```ini
89
+ # GCP Project Configuration (for deployment)
90
+ PROJECT_ID=your-gcp-project-id
91
+ LOCATION=us-central1
92
+
93
+ # Optional: Temporary File Cleanup (seconds) - Default 86400 (24h)
94
+ TMP_MAX_AGE_SECONDS=86400
95
+
96
+ # Optional: HuggingFace Token for Gated Models (Local MedSigLIP)
97
+ HF_TOKEN=your_hf_token
98
+ ```
99
+
100
+ **To obtain `HF_TOKEN` for `google/medsiglip-448`:**
101
+ 1. Create a [Hugging Face account](https://huggingface.co/join).
102
+ 2. Visit the [google/medsiglip-448 model page](https://huggingface.co/google/medsiglip-448) and check if you need to accept a license agreement (gated access).
103
+ 3. Go to your [Settings > Access Tokens](https://huggingface.co/settings/tokens) page.
104
+ 4. Create a new token with **Read** permissions.
105
+ 5. Copy the token and paste it into your `.env` file as `HF_TOKEN`.
106
+
107
+ 5. **Start Dev Container**:
108
+ - Open the folder in VS Code.
109
+ - When prompted, click **"Reopen in Container"** (or run standard command `Dev Containers: Reopen in Container`).
110
+ - VS Code will build the container and install all dependencies defined in `requirements-dev.txt` and `package.json`.
111
+
112
+ Inside the integrated terminal of VS Code (running in the container):
113
+ ```bash
114
+ npm install # If not run automatically
115
+ uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
116
+ ```
117
+ - The API will be available at: http://localhost:8000 (docs at http://localhost:8000/docs/)
118
+ - Frontend: http://localhost:8000/
119
+ - **Debug Mode**: Append `?debug` to the URL (e.g., http://localhost:8000/?debug) to reveal detailed model logs, execution timers, saliency maps, and preprocessing calibration settings.
120
+
121
+ ### 🐳 Running with Docker (Manual)
122
+
123
+ If you prefer to run the container manually (outside VS Code):
124
+
125
+ **1. Build the Image:**
126
+ You MUST pass your `HF_TOKEN` as a build argument to download the gated model.
127
+ ```bash
128
+ # Load token from .env or export it matches your environment
129
+ export HF_TOKEN=your_token_here
130
+ docker build --build-arg HF_TOKEN=$HF_TOKEN -t dermatolog-ai-scan .
131
+ ```
132
+
133
+ **2. Run the Container:**
134
+ Pass the token as an environment variable for runtime checks (optional if baked in, but recommended).
135
+ ```bash
136
+ docker run -p 8000:8000 -e HF_TOKEN=$HF_TOKEN dermatolog-ai-scan
137
+ ```
138
+
139
+ ### 🧪 Running Tests
140
+
141
+ We use `pytest` for unit tests and `playwright` for end-to-end tests.
142
+
143
+ - **Unit Tests**:
144
+ ```bash
145
+ pytest tests/unit
146
+ ```
147
+
148
+ - **Integration/E2E Tests**:
149
+ ```bash
150
+ pytest tests/e2e
151
+ ```
152
+
153
+ - **JavaScript Unit Tests**:
154
+ ```bash
155
+ npm test
156
+ ```
157
+
158
+ ### Deployment
159
+
160
+ The application is containerized and can be deployed to Google Cloud Run, AWS, or Kubernetes.
161
+
162
+ 👉 **See [DEPLOY.md](DEPLOY.md) for full deployment instructions.**
app/__init__.py ADDED
File without changes
app/config.py ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Configuration settings for the Dermatolog AI Scan application.
3
+ Contains model parameters, clinical thresholds, and system constants.
4
+ """
5
+
6
+
7
+ # --- Stage 2: Result Interpretation Parameters ---
8
+
9
+ # Shannon Entropy threshold (in bits) for determining prediction reliability.
10
+ # Entropy measures the model's "confusion" across all classes.
11
+ # For a 10-class distribution:
12
+ # - Max entropy (complete guessing) is ~3.32 bits.
13
+ # - High confidence (90% in one class) approaches 0 bits.
14
+ # Threshold of 2.5 allows for relative clarity but flags high-chaos distributions.
15
+ INTERPRETER_ENTROPY_THRESHOLD = 2.5
16
+
17
+ # Margin threshold specifically for Mixed (Tumor vs Non-Tumor) cases.
18
+ # If the top prediction is a tumor but the second is non-tumor (or vice versa),
19
+ # and the absolute difference in their scores is less than this value,
20
+ # the result is annotated as "Not clear".
21
+ INTERPRETER_MARGIN_THRESHOLD = 0.05
22
+
23
+ # --- Confidence Classification (Margin Based) ---
24
+
25
+ # Mapping of confidence levels based on the margin between Top-1 and Top-2 results.
26
+ # Used to provide qualitative feedback to the end user.
27
+ CONFIDENCE_CLASSES = [
28
+ {"min": 0.40, "label": "Confident", "color_hint": "green"},
29
+ {"min": 0.20, "label": "Plausible", "color_hint": "gray"},
30
+ {"min": 0.10, "label": "Low confidence", "color_hint": "yellow"},
31
+ {"min": 0.00, "label": "Results unclear", "color_hint": "red"},
32
+ ]
33
+
34
+
35
+ # --- Model Configuration ---
36
+
37
+ # The target image resolution for MedSigLIP.
38
+ # Changing this requires a compatible model checkpoint.
39
+ MODEL_IMAGE_SIZE = (448, 448)
40
+
41
+ # The default HuggingFace model path for MedSigLIP.
42
+ MEDSIGLIP_MODEL_NAME = "google/medsiglip-448"
43
+
44
+
app/dal/__init__.py ADDED
File without changes
app/dal/database.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import duckdb
2
+ import os
3
+ import logging
4
+ from contextlib import contextmanager
5
+
6
+ logger = logging.getLogger(__name__)
7
+
8
+ class DuckDBManager:
9
+ def __init__(self, db_path: str = "data/app.duckdb"):
10
+ self.db_path = db_path
11
+ # Initialize or migrate schema
12
+ self._init_schema()
13
+
14
+ def _init_schema(self):
15
+ """Initializes the database schema."""
16
+ try:
17
+ with self.get_connection() as con:
18
+ con.execute("""
19
+ CREATE TABLE IF NOT EXISTS interaction_logs (
20
+ id INTEGER PRIMARY KEY,
21
+ timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
22
+ prompt TEXT,
23
+ response TEXT,
24
+ latency_ms INTEGER
25
+ );
26
+ CREATE SEQUENCE IF NOT EXISTS seq_interaction_id START 1;
27
+
28
+ CREATE TABLE IF NOT EXISTS photos (
29
+ id UUID PRIMARY KEY,
30
+ session_id VARCHAR,
31
+ filename VARCHAR,
32
+ content BLOB,
33
+ creation_date DATE,
34
+ uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
35
+ md5_hash VARCHAR,
36
+ analysis_results VARCHAR,
37
+ analysis_date VARCHAR
38
+ );
39
+ -- Migration for existing tables
40
+ ALTER TABLE photos ADD COLUMN IF NOT EXISTS md5_hash VARCHAR;
41
+ ALTER TABLE photos ADD COLUMN IF NOT EXISTS analysis_results VARCHAR;
42
+ ALTER TABLE photos ADD COLUMN IF NOT EXISTS analysis_date VARCHAR;
43
+ """)
44
+ logger.info("Database schema initialized.")
45
+ except Exception as e:
46
+ logger.error(f"Failed to init schema: {e}")
47
+
48
+ @contextmanager
49
+ def get_connection(self):
50
+ """Yields a DuckDB connection."""
51
+ # DuckDB handles concurrency well, but creating a connection per request is safe for persistence
52
+ con = duckdb.connect(self.db_path)
53
+ try:
54
+ yield con
55
+ finally:
56
+ con.close()
57
+
58
+ def log_interaction(self, prompt: str, response: str, latency_ms: int):
59
+ try:
60
+ with self.get_connection() as con:
61
+ con.execute("""
62
+ INSERT INTO interaction_logs (id, prompt, response, latency_ms)
63
+ VALUES (nextval('seq_interaction_id'), ?, ?, ?)
64
+ """, [prompt, response, latency_ms])
65
+ except Exception as e:
66
+ logger.error(f"Failed to log interaction: {e}")
67
+
68
+ db_manager = DuckDBManager(db_path=os.getenv("DUCKDB_PATH", "data/app.duckdb"))
app/dal/photo_repo.py ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ import logging
3
+ from typing import List, Optional, Tuple, Dict
4
+
5
+ logger = logging.getLogger(__name__)
6
+
7
+ class PhotoRepository:
8
+ def __init__(self):
9
+ # In-memory storage instead of DuckDB
10
+ # key: session_id, value: { photo_id: metadata_dict }
11
+ self._storage: Dict[str, Dict[str, dict]] = {}
12
+
13
+ def _get_session_store(self, session_id: str) -> Dict[str, dict]:
14
+ if session_id not in self._storage:
15
+ self._storage[session_id] = {}
16
+ return self._storage[session_id]
17
+
18
+ def find_duplicate(self, session_id: str, file_hash: str) -> Optional[str]:
19
+ store = self._get_session_store(session_id)
20
+ for photo_id, metadata in store.items():
21
+ if metadata.get("md5_hash") == file_hash:
22
+ return photo_id
23
+ return None
24
+
25
+ def create_photo(self, photo_id: str, session_id: str, filename: str, ext: str, creation_date: str, file_hash: str, content: bytes):
26
+ store = self._get_session_store(session_id)
27
+ store[photo_id] = {
28
+ "id": photo_id,
29
+ "filename": filename,
30
+ "content": content,
31
+ "creation_date": creation_date,
32
+ "uploaded_at": str(logging.Formatter().formatTime(logging.LogRecord(None, None, None, None, None, None, None), "%Y-%m-%d %H:%M:%S")),
33
+ "md5_hash": file_hash,
34
+ "analysis_results": None,
35
+ "analysis_date": None
36
+ }
37
+
38
+ def get_timeline_photos(self, session_id: str) -> List[Tuple]:
39
+ store = self._get_session_store(session_id)
40
+ results = []
41
+ # Convert to the tuple format expected by router
42
+ # (id, filename, creation_date, uploaded_at, analysis_results, analysis_date)
43
+ for p in store.values():
44
+ results.append((
45
+ p["id"],
46
+ p["filename"],
47
+ p["creation_date"],
48
+ p["uploaded_at"],
49
+ p["analysis_results"],
50
+ p["analysis_date"]
51
+ ))
52
+ # Sort by creation_date DESC, then uploaded_at DESC
53
+ return sorted(results, key=lambda x: (x[2], x[3]), reverse=True)
54
+
55
+ def save_analysis_results(self, photo_id: str, session_id: str, results_json: str):
56
+ store = self._get_session_store(session_id)
57
+ if photo_id in store:
58
+ store[photo_id]["analysis_results"] = results_json
59
+ store[photo_id]["analysis_date"] = str(logging.Formatter().formatTime(logging.LogRecord(None, None, None, None, None, None, None), "%H:%M:%S"))
60
+
61
+ def get_analysis_results(self, photo_id: str, session_id: str) -> Optional[Tuple[str, str]]:
62
+ store = self._get_session_store(session_id)
63
+ p = store.get(photo_id)
64
+ if p and p["analysis_results"]:
65
+ return (p["analysis_results"], p["analysis_date"])
66
+ return None
67
+
68
+ def update_date(self, photo_id: str, session_id: str, new_date: str):
69
+ store = self._get_session_store(session_id)
70
+ if photo_id in store:
71
+ store[photo_id]["creation_date"] = new_date
72
+
73
+ def get_photo_metadata(self, photo_id: str, session_id: str) -> Optional[Tuple[str, bytes]]:
74
+ store = self._get_session_store(session_id)
75
+ p = store.get(photo_id)
76
+ if p:
77
+ return (p["filename"], p["content"])
78
+ return None
79
+
80
+ def delete_photo(self, photo_id: str, session_id: str):
81
+ store = self._get_session_store(session_id)
82
+ if photo_id in store:
83
+ del store[photo_id]
84
+
85
+ def clear_session(self, session_id: str):
86
+ if session_id in self._storage:
87
+ del self._storage[session_id]
88
+
89
+ photo_repo = PhotoRepository()
app/dermatology_data.py ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Comprehensive dermatology labels based on EU prevalence and referral guidelines
2
+ # Rationale:
3
+ # 1. Malignant/Pre-malignant: Detecting high-mortality (Melanoma) and high-prevalence (BCC/SCC) cancers is the priority.
4
+ # 2. Inflammatory: Eczema, Psoriasis, and Acne are the most common burdens on quality of life in EU.
5
+ # 3. Infectious: Fungal and viral infections are frequent reasons for primary care visits.
6
+ # 4. Benign: Essential for differential diagnosis to reduce unnecessary anxiety or referrals.
7
+
8
+ MEDSIGLIP_DERMATOLOGY_LABELS = {
9
+ # Malignant & Pre-malignant
10
+ "Melanoma": "malignant melanoma, asymmetric pigmented lesion with irregular borders and color variegation",
11
+ "Basal Cell Carcinoma": "basal cell carcinoma, pearly translucent papule with arborizing telangiectasia",
12
+ "Squamous Cell Carcinoma": "squamous cell carcinoma, indurated hyperkeratotic erythematous nodule or ulcerated plaque",
13
+ "Actinic Keratosis": "actinic keratosis, rough scaly erythematous macule on sun-damaged skin",
14
+ "Bowen's Disease": "Bowen's disease, well-demarcated erythematous scaly plaque",
15
+ "Dysplastic Nevus": "dysplastic nevus, atypical melanocytic lesion with irregular borders and variable pigmentation",
16
+
17
+ # Benign Tumors (Differential Diagnosis)
18
+ "Melanocytic Nevus": "benign melanocytic nevus, well-circumscribed symmetrical pigmented macule",
19
+ "Seborrheic Keratosis": "seborrheic keratosis, sharply demarcated verrucous plaque with stuck-on appearance",
20
+ "Dermatofibroma": "dermatofibroma, firm hyperpigmented dermal nodule with positive dimple sign",
21
+ "Haemangioma": "hemangioma, benign vascular anomaly, bright red or violaceous nodule",
22
+ "Epidermoid Cyst": "epidermoid cyst, subcutaneous skin-colored nodule with central punctum",
23
+
24
+ # Inflammatory Conditions
25
+ "Psoriasis": "psoriasis vulgaris, well-demarcated erythematous plaques with thick silvery-white scale",
26
+ "Atopic Dermatitis": "atopic dermatitis, pruritic erythematous scaling patches with lichenification",
27
+ "Acne Vulgaris": "acne vulgaris, inflammatory eruption with comedones, papules, and pustules",
28
+ "Rosacea": "rosacea, facial erythema and telangiectasia with inflammatory papules",
29
+ "Urticaria": "urticaria, transient circumscribed erythematous and edematous wheals",
30
+ "Lichen Planus": "lichen planus, pruritic purple polygonal planar papules with Wickham striae",
31
+ "Hidradenitis Suppurativa": "hidradenitis suppurativa, painful deep-seated inflammatory nodules and abscesses",
32
+
33
+ # Infectious
34
+ "Fungal Infection": "tinea fungal infection, an annular, scaling, erythematous patch with raised borders and central clearing",
35
+ "Herpes Zoster": "herpes zoster, a unilateral, dermatomal eruption of grouped, painful vesicles on an erythematous base",
36
+ "Impetigo": "impetigo, superficial bacterial infection with erosions and classic honey-colored crusting",
37
+ "Warts": "verruca vulgaris, a viral infection presenting as a hyperkeratotic, exophytic papule",
38
+ "Molluscum Contagiosum": "molluscum contagiosum, presenting as firm, dome-shaped, umbilicated, pearly papules",
39
+
40
+ # Pigmentary & Hair
41
+ "Vitiligo": "vitiligo, depigmented white macules and patches devoid of melanocytes",
42
+ "Alopecia Areata": "alopecia areata, localized patches of non-scarring hair loss on the scalp or body",
43
+ "Melasma": "melasma, symmetric, hyperpigmented brown macules primarily on sun-exposed facial areas",
44
+
45
+ # Miscellaneous
46
+ "Insect Bites": "arthropod bite reaction, intensely pruritic, erythematous papules with a central punctum",
47
+ "Folliculitis": "folliculitis, inflammation of hair follicles with multiple erythematous papules and pustules",
48
+ "Drug Rash": "morbilliform drug eruption, a generalized, symmetric, maculopapular erythematous exanthem",
49
+
50
+ # Baseline
51
+ "Normal Skin": "normal, healthy skin with intact epidermis, uniform texture, and no visible lesions"
52
+ }
53
+ #Inflammatory vs. Neoplastic Differentiation: The model can effectively distinguish
54
+ # between inflammatory skin conditions and neoplastic (cancerous)
55
+ ## Used for triage analysis
56
+ MEDSIGLIP_DERMATOLOGY_FIRST_CLASSES = {
57
+ # 1. Inflammatory
58
+ "Inflammatory skin disease": "showing inflammatory lesion, or a rash or redness, or scaling",
59
+ # 2. Neoplastic
60
+ #"Neoplastic skin tumor": "neoplastic skin tumor or suspect growth or abnormal mole",
61
+ "Melanoma": MEDSIGLIP_DERMATOLOGY_LABELS["Melanoma"],
62
+ # 3. Zero-Shot Baseline
63
+ "Healthly skin": "melanocytic naevus, pigmented naevus"
64
+ }
65
+
66
+ CANCEROUS_TUMOR_CLASSES = {
67
+ "Melanoma",
68
+ "Basal Cell Carcinoma",
69
+ "Squamous Cell Carcinoma",
70
+ "Bowen's Disease"
71
+ }
72
+
73
+ BENIGN_TUMOR_CLASSES = {
74
+ "Melanocytic Nevus",
75
+ "Seborrheic Keratosis",
76
+ "Dermatofibroma",
77
+ "Haemangioma",
78
+ "Epidermoid Cyst"
79
+ }
80
+
81
+ # Narrow set of labels focusing on MedSigLIP's highest performance tiers
82
+ MEDSIGLIP_DERMATOLOGY_NARROW_LABELS = {
83
+ # 1. High-Precision Vascular & Pigmented Lesions
84
+ "Melanoma": MEDSIGLIP_DERMATOLOGY_LABELS["Melanoma"],
85
+ "Basal Cell Carcinoma": MEDSIGLIP_DERMATOLOGY_LABELS["Basal Cell Carcinoma"],
86
+ "Melanocytic Nevus": MEDSIGLIP_DERMATOLOGY_LABELS["Melanocytic Nevus"],
87
+ "Seborrheic Keratosis": MEDSIGLIP_DERMATOLOGY_LABELS["Seborrheic Keratosis"],
88
+
89
+ # 2. Texture-Heavy Inflammatory Conditions
90
+ "Psoriasis": MEDSIGLIP_DERMATOLOGY_LABELS["Psoriasis"],
91
+ "Atopic Dermatitis": MEDSIGLIP_DERMATOLOGY_LABELS["Atopic Dermatitis"],
92
+ "Acne Vulgaris": MEDSIGLIP_DERMATOLOGY_LABELS["Acne Vulgaris"],
93
+ "Rosacea": MEDSIGLIP_DERMATOLOGY_LABELS["Rosacea"],
94
+
95
+ # 3. Morphologically Distinct Infections
96
+ "Herpes Zoster": MEDSIGLIP_DERMATOLOGY_LABELS["Herpes Zoster"],
97
+ "Warts": MEDSIGLIP_DERMATOLOGY_LABELS["Warts"],
98
+ "Molluscum Contagiosum": MEDSIGLIP_DERMATOLOGY_LABELS["Molluscum Contagiosum"],
99
+
100
+ # 4. Zero-Shot Baseline
101
+ "Normal Skin": MEDSIGLIP_DERMATOLOGY_LABELS["Normal Skin"]
102
+ }
103
+
104
+
app/main.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import time
2
+ import logging
3
+ import uuid
4
+ import os
5
+ from dotenv import load_dotenv
6
+
7
+ load_dotenv()
8
+
9
+ from fastapi import FastAPI, HTTPException, Request, Response
10
+ from fastapi.staticfiles import StaticFiles
11
+ from fastapi.templating import Jinja2Templates
12
+ from fastapi.responses import HTMLResponse
13
+ from starlette.middleware.base import BaseHTTPMiddleware
14
+
15
+
16
+ from app.models import HealthCheckResponse
17
+ from app.routers.photos import router as photos_router
18
+ from app.routers.api import router as api_router
19
+
20
+ # Configure logging
21
+ logging.basicConfig(level=logging.INFO)
22
+ logger = logging.getLogger(__name__)
23
+
24
+ app = FastAPI(
25
+ title="Dermatolog AI Scan",
26
+ description="FastAPI application for dermatology analysis",
27
+ version="1.0.0"
28
+ )
29
+
30
+ # Simple Session Middleware
31
+ class SessionMiddleware(BaseHTTPMiddleware):
32
+ async def dispatch(self, request: Request, call_next):
33
+ session_id = request.cookies.get("session_id")
34
+ created_new = False
35
+ if not session_id:
36
+ session_id = str(uuid.uuid4())
37
+ created_new = True
38
+ # Hack: Inject into request scope so endpoints can see it if they looked there,
39
+ # but usually they look at cookies. We rely on the client sending it back,
40
+ # but for the *first* request, we need to handle it.
41
+ # Ideally endpoints assume cookie exists.
42
+ # Let's set the cookie on the response.
43
+
44
+ # Pass session_id in request state if needed?
45
+ # request.state.session_id = session_id
46
+
47
+ response = await call_next(request)
48
+
49
+ if created_new:
50
+ # Set cookie for 1 day
51
+ response.set_cookie(key="session_id", value=session_id, max_age=86400)
52
+
53
+ return response
54
+
55
+ app.add_middleware(SessionMiddleware)
56
+
57
+ app.include_router(photos_router)
58
+ app.include_router(api_router)
59
+
60
+ # Mount static files
61
+ BASE_DIR = os.path.dirname(os.path.abspath(__file__))
62
+ app.mount("/static", StaticFiles(directory=os.path.join(BASE_DIR, "static")), name="static")
63
+ templates = Jinja2Templates(directory=os.path.join(BASE_DIR, "templates"))
64
+
65
+ @app.get("/", response_class=HTMLResponse)
66
+ async def read_root(request: Request):
67
+ """Serve the main frontend page."""
68
+ return templates.TemplateResponse("index.html", {"request": request})
app/models.py ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pydantic import BaseModel
2
+ from typing import List, Optional
3
+ class HealthCheckResponse(BaseModel):
4
+ status: str
5
+ yolo_available: bool
6
+
7
+ class Photo(BaseModel):
8
+ id: str
9
+ filename: str
10
+ creation_date: str # ISO date string YYYY-MM-DD
11
+ uploaded_at: str
12
+ analysis: Optional[object] = None # Can be List[dict] (legacy) or dict (new with comparison)
13
+ analysis_date: Optional[str] = None
14
+ local_content: Optional[str] = None # Base64 data for client-side storage
15
+
16
+
17
+ # Response model for the timeline: a list of either Photo (single) or VirtualDirectory (group)
18
+ # In Pydantic V2 we might use Union, but for simplicity/JSON serialization,
19
+ # we can return a list of objects that have a 'type' field.
20
+
21
+ class TimelineItem(BaseModel):
22
+ type: str # 'photo' or 'directory'
23
+ date: str
24
+ data: Optional[Photo] = None # If type is photo
25
+ items: Optional[List[Photo]] = None # If type is directory
26
+
27
+
28
+
29
+ from app.config import INTERPRETER_MARGIN_THRESHOLD
30
+
31
+ class SinglePhotoAnalysisRequest(BaseModel):
32
+ # Default labels for zero-shot classification from centralized config
33
+ candidate_labels: Optional[List[str]] = None
34
+ model: Optional[str] = "medsiglip" # "medsiglip" only now
35
+ base64_image: Optional[str] = None # Client-side image data
36
+ margin_threshold: Optional[float] = INTERPRETER_MARGIN_THRESHOLD
37
+
38
+ class SinglePhotoAnalysisResponse(BaseModel):
39
+ photo_id: str
40
+ predictions: List[dict]
41
+ primary_model_name: Optional[str] = None
42
+ analysis_date: Optional[str] = None
43
+ prepared_image_base64: Optional[str] = None
44
+ saliency_base64: Optional[str] = None # Returning saliency as base64
45
+ interpretation: Optional[dict] = None
46
+ preprocess_strategy: Optional[dict] = None
47
+ execution_times: Optional[dict] = None
48
+
49
+ class SaliencyRequest(BaseModel):
50
+ base64_image: str
51
+ target_label: str
52
+
53
+ class SaliencyResponse(BaseModel):
54
+ photo_id: str
55
+ saliency_base64: str
app/photos.py ADDED
@@ -0,0 +1,334 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import uuid
2
+ import base64
3
+ import logging
4
+ import io
5
+ import json
6
+ import os
7
+ from datetime import datetime, date
8
+ from typing import List, Optional
9
+ from fastapi import APIRouter, UploadFile, File, Form, HTTPException, Cookie, Response, Request
10
+ from fastapi.responses import JSONResponse
11
+ from PIL import Image, ExifTags
12
+
13
+
14
+ from app.models import TimelineItem, Photo, VirtualDirectory, SinglePhotoAnalysisRequest, SinglePhotoAnalysisResponse
15
+ from app.services.medsiglip_service import medsiglip_service
16
+ from app.dal.photo_repo import photo_repo
17
+ from app.dermatology_data import EU_DERMATOLOGY_LABELS
18
+
19
+ router = APIRouter(prefix="/api/photos", tags=["photos"])
20
+
21
+ logger = logging.getLogger(__name__)
22
+
23
+ def get_date_from_image(image_bytes: bytes) -> str:
24
+ """Heuristic to find creation date from EXIF or return today."""
25
+ try:
26
+ image = Image.open(io.BytesIO(image_bytes))
27
+ exif = image._getexif()
28
+ if exif:
29
+ # 36867 is DateTimeOriginal, 306 is DateTime
30
+ for tag_id in [36867, 306]:
31
+ if tag_id in exif:
32
+ date_str = exif[tag_id]
33
+ # Format is usually "YYYY:MM:DD HH:MM:SS"
34
+ try:
35
+ dt = datetime.strptime(date_str, "%Y:%m:%d %H:%M:%S")
36
+ return dt.date().isoformat()
37
+ except ValueError:
38
+ continue
39
+ except Exception as e:
40
+ logger.warning(f"Failed to extract EXIF: {e}")
41
+
42
+ # Fallback to today
43
+ return date.today().isoformat()
44
+
45
+ import hashlib
46
+
47
+ @router.post("/upload")
48
+ async def upload_photos(
49
+ request: Request,
50
+ files: List[UploadFile] = File(...),
51
+ ):
52
+ session_id = request.cookies.get("session_id")
53
+ if not session_id:
54
+ raise HTTPException(status_code=400, detail="No session found - reload page")
55
+
56
+ processed_ids = []
57
+ skipped_count = 0
58
+
59
+ try:
60
+ for file in files:
61
+ content = await file.read()
62
+
63
+ # Calculate MD5 hash
64
+ file_hash = hashlib.md5(content).hexdigest()
65
+
66
+ # Check for duplicate in this session
67
+ existing_id = photo_repo.find_duplicate(session_id, file_hash)
68
+
69
+ if existing_id:
70
+ skipped_count += 1
71
+ continue
72
+
73
+ # Heuristic Date Extraction
74
+ creation_date = get_date_from_image(content)
75
+
76
+ photo_id = str(uuid.uuid4())
77
+
78
+ # Save to filesystem
79
+ session_dir = os.path.join("img", session_id)
80
+ os.makedirs(session_dir, exist_ok=True)
81
+
82
+ # Use original extension or default to .jpg
83
+ ext = os.path.splitext(file.filename)[1]
84
+ if not ext:
85
+ ext = ".jpg"
86
+
87
+ file_path = os.path.join(session_dir, f"{photo_id}{ext}")
88
+ with open(file_path, "wb") as f:
89
+ f.write(content)
90
+
91
+ # Save metadata to DB via Repo
92
+ photo_repo.create_photo(photo_id, session_id, file.filename, ext, creation_date, file_hash)
93
+
94
+ processed_ids.append(photo_id)
95
+
96
+ return {
97
+ "uploaded": len(processed_ids),
98
+ "skipped": skipped_count,
99
+ "ids": processed_ids,
100
+ "message": f"Uploaded {len(processed_ids)} photos, skipped {skipped_count} duplicates."
101
+ }
102
+
103
+ except Exception as e:
104
+ logger.error(f"Upload failed: {e}")
105
+ raise HTTPException(status_code=500, detail=str(e))
106
+
107
+ @router.get("", response_model=List[TimelineItem])
108
+ async def get_timeline(request: Request):
109
+ session_id = request.cookies.get("session_id")
110
+ if not session_id:
111
+ return []
112
+
113
+ try:
114
+ # Fetch from Repo
115
+ rows = photo_repo.get_timeline_photos(session_id)
116
+
117
+ photos = []
118
+ for r in rows:
119
+ analysis_data = None
120
+ if len(r) > 4 and r[4]:
121
+ try:
122
+ analysis_data = json.loads(r[4])
123
+ except:
124
+ pass
125
+
126
+ analysis_date = None
127
+ if len(r) > 5 and r[5]:
128
+ analysis_date = r[5]
129
+
130
+ photos.append(Photo(
131
+ id=str(r[0]),
132
+ filename=r[1],
133
+ creation_date=r[2],
134
+ uploaded_at=r[3],
135
+ analysis=analysis_data,
136
+ analysis_date=analysis_date
137
+ ))
138
+
139
+ # Grouping Logic: ALWAYS group by date (directory mode)
140
+ timeline = []
141
+ if not photos:
142
+ return timeline
143
+
144
+ current_group = []
145
+ current_date = None
146
+
147
+ for p in photos:
148
+ if p.creation_date != current_date:
149
+ # Flush previous group
150
+ if current_group:
151
+ timeline.append(TimelineItem(
152
+ type="directory",
153
+ date=current_date,
154
+ items=current_group
155
+ ))
156
+ # Start new group
157
+ current_group = [p]
158
+ current_date = p.creation_date
159
+ else:
160
+ current_group.append(p)
161
+
162
+ # Flush last group
163
+ if current_group:
164
+ timeline.append(TimelineItem(
165
+ type="directory",
166
+ date=current_date,
167
+ items=current_group
168
+ ))
169
+
170
+ logger.info(f"Timeline fetched: {len(timeline)} groups for session {session_id}")
171
+ return timeline
172
+
173
+ except Exception as e:
174
+ logger.error(f"Timeline fetch failed: {e}")
175
+ raise HTTPException(status_code=500, detail=str(e))
176
+
177
+ def _append_group(timeline: List[TimelineItem], group: List[Photo], date_str: str):
178
+ if len(group) == 1:
179
+ # Single photo item
180
+ timeline.append(TimelineItem(
181
+ type="photo",
182
+ date=date_str,
183
+ data=group[0]
184
+ ))
185
+ else:
186
+ # Virtual Directory
187
+ timeline.append(TimelineItem(
188
+ type="directory",
189
+ date=date_str,
190
+ items=group
191
+ ))
192
+
193
+ @router.patch("/{photo_id}/date")
194
+ async def patch_photo_date(photo_id: str, request: Request, payload: dict):
195
+ # payload: {"date": "2023-01-01"}
196
+ session_id = request.cookies.get("session_id")
197
+ new_date = payload.get("date")
198
+
199
+ if not new_date:
200
+ raise HTTPException(status_code=400, detail="Date required")
201
+
202
+ try:
203
+ photo_repo.update_date(photo_id, session_id, new_date)
204
+ return {"status": "updated"}
205
+ except Exception as e:
206
+ logger.error(f"Update failed: {e}")
207
+ raise HTTPException(status_code=500, detail=str(e))
208
+
209
+ @router.get("/{photo_id}/content")
210
+ async def get_photo_content(photo_id: str, request: Request):
211
+ session_id = request.cookies.get("session_id")
212
+ try:
213
+ result = photo_repo.get_photo_metadata(photo_id, session_id)
214
+ if not result:
215
+ raise HTTPException(status_code=404, detail="Photo not found")
216
+
217
+ original_filename = result[0]
218
+ stored_content = result[1]
219
+
220
+ try:
221
+ local_filename = stored_content.decode('utf-8')
222
+ file_path = os.path.join("img", session_id, local_filename)
223
+
224
+ if os.path.exists(file_path):
225
+ with open(file_path, "rb") as f:
226
+ content = f.read()
227
+ else:
228
+ content = stored_content
229
+ except:
230
+ content = stored_content
231
+
232
+ # Simple mimetype detection or default
233
+ media_type = "image/jpeg"
234
+ if original_filename.lower().endswith(".png"):
235
+ media_type = "image/png"
236
+
237
+ return Response(content=content, media_type=media_type)
238
+
239
+ except Exception as e:
240
+ logger.error(f"Content fetch failed: {e}")
241
+ raise HTTPException(status_code=500, detail=str(e))
242
+
243
+ @router.post("/{photo_id}/analyze", response_model=SinglePhotoAnalysisResponse)
244
+ async def analyze_photo(photo_id: str, request: Request, payload: SinglePhotoAnalysisRequest):
245
+ session_id = request.cookies.get("session_id")
246
+ if not session_id:
247
+ raise HTTPException(status_code=400, detail="No session found")
248
+
249
+ try:
250
+ # Check cache first
251
+ cached = photo_repo.get_analysis_results(photo_id, session_id)
252
+ if cached:
253
+ try:
254
+ # cached is (json_str, date_str)
255
+ preds = json.loads(cached[0])
256
+ return SinglePhotoAnalysisResponse(
257
+ photo_id=photo_id,
258
+ predictions=preds,
259
+ analysis_date=cached[1]
260
+ )
261
+ except Exception:
262
+ pass
263
+
264
+ # 1. Fetch Photo Content via Repo
265
+ result = photo_repo.get_photo_metadata(photo_id, session_id)
266
+ if not result:
267
+ raise HTTPException(status_code=404, detail="Photo not found")
268
+
269
+ stored_content = result[1]
270
+
271
+ try:
272
+ local_filename = stored_content.decode('utf-8')
273
+ file_path = os.path.join("img", session_id, local_filename)
274
+ if os.path.exists(file_path):
275
+ with open(file_path, "rb") as f:
276
+ content = f.read()
277
+ else:
278
+ content = stored_content
279
+ except:
280
+ content = stored_content
281
+
282
+ # 2. Run Inference
283
+ labels = payload.candidate_labels
284
+ if not labels:
285
+ labels = EU_DERMATOLOGY_LABELS
286
+
287
+ predictions = medsiglip_service.get_embeddings(content, texts=labels)
288
+
289
+ # Save results for future use
290
+ try:
291
+ photo_repo.save_analysis_results(photo_id, session_id, json.dumps(predictions))
292
+ except Exception as e:
293
+ logger.error(f"Failed to save analysis results: {e}")
294
+
295
+ return SinglePhotoAnalysisResponse(
296
+ photo_id=photo_id,
297
+ predictions=predictions,
298
+ analysis_date=datetime.now().isoformat()
299
+ )
300
+
301
+ except Exception as e:
302
+ logger.error(f"Analysis failed: {e}")
303
+ raise HTTPException(status_code=500, detail=str(e))
304
+
305
+ @router.delete("/{photo_id}")
306
+ async def delete_photo(photo_id: str, request: Request):
307
+ session_id = request.cookies.get("session_id")
308
+ if not session_id:
309
+ raise HTTPException(status_code=400, detail="No session found")
310
+
311
+ try:
312
+ photo_repo.delete_photo(photo_id, session_id)
313
+ # Ideally delete file too, but keeping it simple for now
314
+ return {"status": "deleted", "id": photo_id}
315
+
316
+ except Exception as e:
317
+ logger.error(f"Delete failed: {e}")
318
+ raise HTTPException(status_code=500, detail=str(e))
319
+
320
+ @router.delete("")
321
+ async def clear_session_photos(request: Request):
322
+ """Deletes all photos associated with the current session ID."""
323
+ session_id = request.cookies.get("session_id")
324
+ if not session_id:
325
+ raise HTTPException(status_code=400, detail="No session found")
326
+
327
+ try:
328
+ photo_repo.clear_session(session_id)
329
+ # Ideally clean up directory
330
+ return {"status": "cleared", "message": "All session photos deleted"}
331
+
332
+ except Exception as e:
333
+ logger.error(f"Clear session failed: {e}")
334
+ raise HTTPException(status_code=500, detail=str(e))
app/routers/__init__.py ADDED
File without changes
app/routers/api.py ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import APIRouter
2
+ from app.models import HealthCheckResponse
3
+ import os
4
+
5
+ router = APIRouter(prefix="/api")
6
+
7
+ from app.services.medsiglip_service import medsiglip_service
8
+ from app.services.yolo_service import yolo_service
9
+
10
+ @router.get("/health", response_model=HealthCheckResponse)
11
+ async def health_check():
12
+ """Health check endpoint."""
13
+ yolo_available = yolo_service.load_model() is not None
14
+ return HealthCheckResponse(
15
+ status="OK",
16
+ yolo_available=yolo_available
17
+ )
app/routers/photos.py ADDED
@@ -0,0 +1,394 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import uuid
2
+ import base64
3
+ import logging
4
+ import io
5
+ import json
6
+ import os
7
+ import time
8
+ from datetime import datetime, date
9
+ from typing import List, Optional
10
+ from fastapi import APIRouter, UploadFile, File, HTTPException, Response, Request
11
+ from PIL import Image
12
+
13
+
14
+ from app.models import TimelineItem, Photo, SinglePhotoAnalysisRequest, SinglePhotoAnalysisResponse, SaliencyRequest, SaliencyResponse
15
+ from app.services.medsiglip_service import medsiglip_service
16
+ from app.services.medsiglip_modality_wrapper import (
17
+ medsiglip_wrapped_service,
18
+ )
19
+ from app.services.image_preprocess_service import image_preprocess_service, PreprocessStrategy
20
+ from app.services.result_interpreter import result_interpreter
21
+ from app.dal.photo_repo import photo_repo
22
+
23
+ router = APIRouter(prefix="/api/photos", tags=["photos"])
24
+
25
+ logger = logging.getLogger(__name__)
26
+
27
+ def get_date_from_image(image_bytes: bytes) -> str:
28
+ """Heuristic to find creation date from EXIF or return today."""
29
+ try:
30
+ image = Image.open(io.BytesIO(image_bytes))
31
+ exif = image._getexif()
32
+ if exif:
33
+ # 36867 is DateTimeOriginal, 306 is DateTime
34
+ for tag_id in [36867, 306]:
35
+ if tag_id in exif:
36
+ date_str = exif[tag_id]
37
+ # Format is usually "YYYY:MM:DD HH:MM:SS"
38
+ try:
39
+ dt = datetime.strptime(date_str, "%Y:%m:%d %H:%M:%S")
40
+ return dt.date().isoformat()
41
+ except ValueError:
42
+ continue
43
+ except Exception as e:
44
+ logger.warning(f"Failed to extract EXIF: {e}")
45
+
46
+ # Fallback to today
47
+ return date.today().isoformat()
48
+
49
+ import hashlib
50
+
51
+ @router.post("/upload")
52
+ async def upload_photos(
53
+ request: Request,
54
+ files: List[UploadFile] = File(...),
55
+ ):
56
+ session_id = request.cookies.get("session_id")
57
+ if not session_id:
58
+ raise HTTPException(status_code=400, detail="No session found - reload page")
59
+
60
+ processed_ids = []
61
+ skipped_count = 0
62
+
63
+ try:
64
+ for file in files:
65
+ content = await file.read()
66
+
67
+ # Calculate MD5 hash
68
+ file_hash = hashlib.md5(content).hexdigest()
69
+
70
+ # Check for duplicate in this session
71
+ existing_id = photo_repo.find_duplicate(session_id, file_hash)
72
+
73
+ if existing_id:
74
+ skipped_count += 1
75
+ continue
76
+
77
+ # Heuristic Date Extraction
78
+ creation_date = get_date_from_image(content)
79
+
80
+ photo_id = str(uuid.uuid4())
81
+
82
+ # Use original extension or default to .jpg
83
+ ext = os.path.splitext(file.filename)[1]
84
+ if not ext:
85
+ ext = ".jpg"
86
+
87
+ # Save metadata and binary content to Repo
88
+ photo_repo.create_photo(photo_id, session_id, file.filename, ext, creation_date, file_hash, content)
89
+
90
+ processed_ids.append(photo_id)
91
+
92
+ return {
93
+ "uploaded": len(processed_ids),
94
+ "skipped": skipped_count,
95
+ "ids": processed_ids,
96
+ "message": f"Uploaded {len(processed_ids)} photos, skipped {skipped_count} duplicates."
97
+ }
98
+
99
+ except Exception as e:
100
+ logger.error(f"Upload failed: {e}")
101
+ raise HTTPException(status_code=500, detail=str(e))
102
+
103
+ @router.get("", response_model=List[TimelineItem])
104
+ async def get_timeline(request: Request):
105
+ session_id = request.cookies.get("session_id")
106
+ if not session_id:
107
+ return []
108
+
109
+ try:
110
+ # Fetch from Repo
111
+ rows = photo_repo.get_timeline_photos(session_id)
112
+
113
+ photos = []
114
+ for r in rows:
115
+ analysis_data = None
116
+ if len(r) > 4 and r[4]:
117
+ try:
118
+ analysis_data = json.loads(r[4])
119
+ except:
120
+ pass
121
+
122
+ analysis_date = None
123
+ if len(r) > 5 and r[5]:
124
+ analysis_date = r[5]
125
+
126
+ photos.append(Photo(
127
+ id=str(r[0]),
128
+ filename=r[1],
129
+ creation_date=r[2],
130
+ uploaded_at=r[3],
131
+ analysis=analysis_data,
132
+ analysis_date=analysis_date
133
+ ))
134
+
135
+ # Grouping Logic: ALWAYS group by date (directory mode)
136
+ timeline = []
137
+ if not photos:
138
+ return timeline
139
+
140
+ current_group = []
141
+ current_date = None
142
+
143
+ for p in photos:
144
+ if p.creation_date != current_date:
145
+ # Flush previous group
146
+ if current_group:
147
+ timeline.append(TimelineItem(
148
+ type="directory",
149
+ date=current_date,
150
+ items=current_group
151
+ ))
152
+ # Start new group
153
+ current_group = [p]
154
+ current_date = p.creation_date
155
+ else:
156
+ current_group.append(p)
157
+
158
+ # Flush last group
159
+ if current_group:
160
+ timeline.append(TimelineItem(
161
+ type="directory",
162
+ date=current_date,
163
+ items=current_group
164
+ ))
165
+
166
+ logger.info(f"Timeline fetched: {len(timeline)} groups for session {session_id}")
167
+ return timeline
168
+
169
+ except Exception as e:
170
+ logger.error(f"Timeline fetch failed: {e}")
171
+ raise HTTPException(status_code=500, detail=str(e))
172
+
173
+ def _append_group(timeline: List[TimelineItem], group: List[Photo], date_str: str):
174
+ if len(group) == 1:
175
+ # Single photo item
176
+ timeline.append(TimelineItem(
177
+ type="photo",
178
+ date=date_str,
179
+ data=group[0]
180
+ ))
181
+ else:
182
+ # Virtual Directory
183
+ timeline.append(TimelineItem(
184
+ type="directory",
185
+ date=date_str,
186
+ items=group
187
+ ))
188
+
189
+ @router.patch("/{photo_id}/date")
190
+ async def patch_photo_date(photo_id: str, request: Request, payload: dict):
191
+ # payload: {"date": "2023-01-01"}
192
+ session_id = request.cookies.get("session_id")
193
+ new_date = payload.get("date")
194
+
195
+ if not new_date:
196
+ raise HTTPException(status_code=400, detail="Date required")
197
+
198
+ try:
199
+ photo_repo.update_date(photo_id, session_id, new_date)
200
+ return {"status": "updated"}
201
+ except Exception as e:
202
+ logger.error(f"Update failed: {e}")
203
+ raise HTTPException(status_code=500, detail=str(e))
204
+
205
+ @router.get("/{photo_id}/content")
206
+ async def get_photo_content(photo_id: str, request: Request):
207
+ session_id = request.cookies.get("session_id")
208
+ try:
209
+ result = photo_repo.get_photo_metadata(photo_id, session_id)
210
+ if not result:
211
+ raise HTTPException(status_code=404, detail="Photo not found")
212
+
213
+ original_filename = result[0]
214
+ stored_content = result[1]
215
+
216
+ content = stored_content
217
+
218
+ # Simple mimetype detection or default
219
+ media_type = "image/jpeg"
220
+ if original_filename.lower().endswith(".png"):
221
+ media_type = "image/png"
222
+
223
+ return Response(content=content, media_type=media_type)
224
+
225
+ except Exception as e:
226
+ logger.error(f"Content fetch failed: {e}")
227
+ raise HTTPException(status_code=500, detail=str(e))
228
+
229
+ @router.post("/{photo_id}/analyze", response_model=SinglePhotoAnalysisResponse)
230
+ async def analyze_photo(photo_id: str, request: Request, payload: SinglePhotoAnalysisRequest):
231
+ session_id = request.cookies.get("session_id")
232
+ if not session_id:
233
+ raise HTTPException(status_code=400, detail="No session found")
234
+
235
+ try:
236
+ # 1. Get Photo Content (Prioritize payload for local-only storage)
237
+ if payload.base64_image:
238
+ # Decode base64 image
239
+ if "," in payload.base64_image:
240
+ _, encoded = payload.base64_image.split(",", 1)
241
+ else:
242
+ encoded = payload.base64_image
243
+ content = base64.b64decode(encoded)
244
+ else:
245
+ # Fallback to fetching from Repo (Database/Filesystem)
246
+ result = photo_repo.get_photo_metadata(photo_id, session_id)
247
+ if not result:
248
+ raise HTTPException(status_code=404, detail="Photo not found")
249
+
250
+ stored_content = result[1]
251
+ content = stored_content
252
+
253
+ # 2. Run Inference
254
+ custom_labels = payload.candidate_labels
255
+
256
+ execution_times = {}
257
+
258
+ # Determine Preprocessing Strategy and prepare image
259
+ start_time = time.perf_counter()
260
+ prep_strategy = image_preprocess_service.recommend_prep_strategy(content)
261
+ prepared_base64 = image_preprocess_service.prepare_image_base64(content)
262
+ execution_times["image_preprocess"] = f"{(time.perf_counter() - start_time):.3f}s"
263
+
264
+ primary_results = []
265
+ primary_name = None
266
+
267
+ # Run Primary (MedSigLIP)
268
+ interpretation = None
269
+ try:
270
+ start_time = time.perf_counter()
271
+ primary_results = medsiglip_wrapped_service.analyze_image(content, custom_labels=custom_labels)
272
+ execution_times["primary_medsiglip"] = f"{(time.perf_counter() - start_time):.3f}s"
273
+ primary_name = medsiglip_wrapped_service.service.model_name
274
+
275
+ # Interpret results with configurable threshold
276
+ interpretation = result_interpreter.interpret(
277
+ primary_results,
278
+ margin_threshold=payload.margin_threshold
279
+ )
280
+ except Exception as e:
281
+ logger.error(f"Primary inference failed: {e}")
282
+ raise HTTPException(status_code=500, detail="Primary model failed")
283
+
284
+ if primary_results:
285
+ logger.info(f"Primary ({primary_name}) top result: {primary_results[0]['label']} ({primary_results[0]['score']:.2f})")
286
+
287
+ results_dict = {
288
+ "primary": primary_results,
289
+ "interpretation": interpretation,
290
+ "primary_model_name": primary_name,
291
+ "preprocess_strategy": prep_strategy,
292
+ "prepared_image_base64": prepared_base64,
293
+ "execution_times": execution_times
294
+ }
295
+
296
+ # Merge with existing cache
297
+ try:
298
+ current_cache = photo_repo.get_analysis_results(photo_id, session_id)
299
+ if current_cache:
300
+ start_data = json.loads(current_cache[0])
301
+ if isinstance(start_data, dict):
302
+ if not primary_results and "primary" in start_data:
303
+ results_dict["primary"] = start_data["primary"]
304
+ results_dict["primary_model_name"] = start_data.get("primary_model_name")
305
+ except:
306
+ pass
307
+
308
+ # Save results
309
+ if not payload.base64_image:
310
+ try:
311
+ photo_repo.save_analysis_results(photo_id, session_id, json.dumps(results_dict))
312
+ except Exception as e:
313
+ logger.error(f"Failed to save analysis results: {e}")
314
+
315
+ return SinglePhotoAnalysisResponse(
316
+ photo_id=photo_id,
317
+ predictions=results_dict.get("primary") or [],
318
+ interpretation=results_dict.get("interpretation"),
319
+ primary_model_name=results_dict.get("primary_model_name"),
320
+ analysis_date=datetime.now().isoformat(),
321
+ prepared_image_base64=results_dict.get("prepared_image_base64"),
322
+ preprocess_strategy=results_dict.get("preprocess_strategy"),
323
+ execution_times=results_dict.get("execution_times")
324
+ )
325
+
326
+ except Exception as e:
327
+ logger.error(f"Analysis failed: {e}")
328
+ raise HTTPException(status_code=500, detail=str(e))
329
+
330
+ @router.delete("/{photo_id}")
331
+ async def delete_photo(photo_id: str, request: Request):
332
+ session_id = request.cookies.get("session_id")
333
+ if not session_id:
334
+ raise HTTPException(status_code=400, detail="No session found")
335
+
336
+ try:
337
+ photo_repo.delete_photo(photo_id, session_id)
338
+ # Ideally delete file too, but keeping it simple for now
339
+ return {"status": "deleted", "id": photo_id}
340
+
341
+ except Exception as e:
342
+ logger.error(f"Delete failed: {e}")
343
+ raise HTTPException(status_code=500, detail=str(e))
344
+
345
+ from app.services.gradcam_service import gradcam_service
346
+
347
+ @router.post("/{photo_id}/saliency", response_model=SaliencyResponse)
348
+ async def generate_saliency_map(
349
+ photo_id: str,
350
+ payload: SaliencyRequest,
351
+ request: Request
352
+ ):
353
+ session_id = request.cookies.get("session_id")
354
+ if not session_id:
355
+ raise HTTPException(status_code=400, detail="No session found")
356
+
357
+ try:
358
+ # Decode base64 image
359
+ if "," in payload.base64_image:
360
+ _, encoded = payload.base64_image.split(",", 1)
361
+ else:
362
+ encoded = payload.base64_image
363
+ content = base64.b64decode(encoded)
364
+
365
+ # Generate Saliency (Grad-CAM)
366
+ heatmap_bytes = gradcam_service.get_heatmap(content, payload.target_label)
367
+ saliency_base64 = base64.b64encode(heatmap_bytes).decode('utf-8')
368
+
369
+ return SaliencyResponse(
370
+ photo_id=photo_id,
371
+ saliency_base64=saliency_base64
372
+ )
373
+
374
+ except Exception as e:
375
+ logger.error(f"Saliency generation failed: {e}")
376
+ raise HTTPException(status_code=500, detail=str(e))
377
+
378
+
379
+ @router.delete("")
380
+ async def clear_session_photos(request: Request):
381
+ """Deletes all photos associated with the current session ID."""
382
+ session_id = request.cookies.get("session_id")
383
+ if not session_id:
384
+ raise HTTPException(status_code=400, detail="No session found")
385
+
386
+ try:
387
+ photo_repo.clear_session(session_id)
388
+
389
+
390
+ return {"status": "cleared", "message": "All session photos deleted"}
391
+
392
+ except Exception as e:
393
+ logger.error(f"Clear session failed: {e}")
394
+ raise HTTPException(status_code=500, detail=str(e))
app/services/__init__.py ADDED
File without changes
app/services/detection_visualizer_service.py ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ from PIL import Image, ImageDraw
3
+ import io
4
+ import numpy as np
5
+
6
+ from app.services.yolo_service import yolo_service
7
+
8
+ logger = logging.getLogger(__name__)
9
+
10
+ class DetectionVisualizerService:
11
+ def __init__(self):
12
+ pass
13
+
14
+ def get_detection_visual(self, image_content: bytes, target_label: str = None) -> bytes:
15
+ """
16
+ Detects lesions using YOLOv8-Nano and draws a bounding box.
17
+ Returns the image with box as bytes (JPEG).
18
+ """
19
+ try:
20
+ # Prepare Inputs
21
+ image = Image.open(io.BytesIO(image_content)).convert("RGB")
22
+
23
+ model = yolo_service.load_model()
24
+ results = model.predict(image, conf=0.25, verbose=False)
25
+
26
+ # Draw on image
27
+ draw = ImageDraw.Draw(image)
28
+
29
+ found = False
30
+ if results and len(results[0].boxes) > 0:
31
+ for box in results[0].boxes:
32
+ b = box.xyxy[0].cpu().numpy()
33
+ conf = float(box.conf[0])
34
+
35
+ # Draw red box for lesion
36
+ draw.rectangle([b[0], b[1], b[2], b[3]], outline="red", width=5)
37
+ # Draw label background
38
+ label = f"Lesion {conf:.2f}"
39
+ draw.text((b[0] + 5, b[1] + 5), label, fill="red")
40
+ found = True
41
+
42
+ if not found:
43
+ # Optional: draw some indicator that nothing was found?
44
+ # Or just return original image.
45
+ pass
46
+
47
+ # Return
48
+ buf = io.BytesIO()
49
+ image.save(buf, format="JPEG")
50
+ return buf.getvalue()
51
+
52
+ except Exception as e:
53
+ logger.error(f"YOLO visualizer error: {e}")
54
+ return image_content
55
+
56
+ detection_visualizer_service = DetectionVisualizerService()
app/services/gradcam_service.py ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import torch
3
+ import torch.nn.functional as F
4
+ import numpy as np
5
+ import cv2
6
+ from PIL import Image
7
+ import io
8
+ from app.services.medsiglip_service import medsiglip_service
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+ class GradCAMService:
13
+ def __init__(self):
14
+ self.gradients = None
15
+ self.activations = None
16
+ self.hooks = []
17
+
18
+ def _save_gradient(self, _module, _grad_input, grad_output):
19
+ self.gradients = grad_output[0]
20
+
21
+ def _save_activation(self, _module, _input, output):
22
+ if isinstance(output, tuple):
23
+ self.activations = output[0]
24
+ else:
25
+ self.activations = output
26
+
27
+ def get_heatmap(self, image_content: bytes, target_label: str) -> bytes:
28
+ """
29
+ Generates a Grad-CAM heatmap for the given image and target label.
30
+ Returns the overlay image as bytes (JPEG).
31
+ """
32
+ # Ensure model is ready
33
+ medsiglip_service._load_model()
34
+ model = medsiglip_service.model
35
+ processor = medsiglip_service.processor
36
+ device = medsiglip_service.device
37
+
38
+ # Clean state
39
+ self.gradients = None
40
+ self.activations = None
41
+ for h in self.hooks: h.remove()
42
+ self.hooks = []
43
+
44
+ try:
45
+ # Prepare Inputs
46
+ image = Image.open(io.BytesIO(image_content)).convert("RGB")
47
+ inputs = processor(text=[target_label], images=image, return_tensors="pt", padding="max_length").to(device)
48
+
49
+ # Hook Target Layer: Last Encoder Layer of Vision Model
50
+ target_layer = model.vision_model.encoder.layers[-1]
51
+
52
+ h1 = target_layer.register_forward_hook(self._save_activation)
53
+ h2 = target_layer.register_full_backward_hook(self._save_gradient)
54
+ self.hooks.extend([h1, h2])
55
+
56
+ # Forward Pass
57
+ model.zero_grad()
58
+ outputs = model(**inputs)
59
+
60
+ # Calculate Score
61
+ score = outputs.logits_per_image[0, 0]
62
+
63
+ # Backward Pass
64
+ score.backward()
65
+
66
+ if self.gradients is None or self.activations is None:
67
+ logger.error("Failed to capture gradients or activations.")
68
+ return image_content
69
+
70
+ # CPU processing
71
+ gradients = self.gradients[0].detach().cpu()
72
+ activations = self.activations[0].detach().cpu()
73
+
74
+ weights = torch.mean(gradients, dim=0)
75
+ cam = torch.matmul(activations, weights)
76
+
77
+ seq_len = cam.shape[0]
78
+ grid_size = int(seq_len**0.5)
79
+
80
+ if grid_size * grid_size != seq_len:
81
+ logger.warning(f"Non-square sequence length: {seq_len}")
82
+ return image_content
83
+
84
+ cam_map = cam.view(grid_size, grid_size)
85
+ cam_map = F.relu(cam_map)
86
+
87
+ if cam_map.max() > 0:
88
+ cam_map = cam_map - cam_map.min()
89
+ cam_map = cam_map / cam_map.max()
90
+
91
+ cam_map_np = cam_map.numpy()
92
+
93
+ img_np = np.array(image)
94
+ heatmap = cv2.resize(cam_map_np, (img_np.shape[1], img_np.shape[0]))
95
+
96
+ heatmap = np.uint8(255 * heatmap)
97
+ heatmap_color = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET)
98
+
99
+ overlay = cv2.addWeighted(img_np, 0.6, heatmap_color, 0.4, 0)
100
+
101
+ out_img = Image.fromarray(overlay)
102
+ buf = io.BytesIO()
103
+ out_img.save(buf, format="JPEG")
104
+ return buf.getvalue()
105
+
106
+ except Exception as e:
107
+ logger.error(f"Grad-CAM error: {e}")
108
+ return image_content
109
+ finally:
110
+ for h in self.hooks: h.remove()
111
+ self.hooks = []
112
+
113
+ gradcam_service = GradCAMService()
app/services/image_preprocess_service.py ADDED
@@ -0,0 +1,192 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import functools
3
+ import time
4
+ import numpy as np
5
+ from PIL import Image
6
+ import io
7
+ from app.services.yolo_service import yolo_service
8
+
9
+ logger = logging.getLogger(__name__)
10
+
11
+ from enum import Enum
12
+
13
+ class PreprocessStrategy(str, Enum):
14
+ CROP = "crop"
15
+ PAD = "pad"
16
+ NONE = "none"
17
+
18
+ class ImagePreprocessService:
19
+ def __init__(self):
20
+ pass
21
+
22
+
23
+ def get_lesion_bbox(self, image_content: bytes, threshold: float = 0.25) -> tuple:
24
+ """
25
+ Detects the lesion bounding box using YOLOv8-Nano.
26
+ """
27
+ try:
28
+ with Image.open(io.BytesIO(image_content)) as img:
29
+ if img.mode != "RGB":
30
+ img = img.convert("RGB")
31
+ width, height = img.size
32
+
33
+ model = yolo_service.load_model()
34
+ if model is None:
35
+ return (0, 0, width, height)
36
+
37
+ # Run inference
38
+ results = model.predict(img, conf=threshold, verbose=False)
39
+
40
+ if not results or len(results[0].boxes) == 0:
41
+ logger.debug("YOLO detection found no boxes, falling back to full image")
42
+ return (0, 0, width, height)
43
+
44
+ # Take the highest confidence box (YOLO sorts by confidence by default)
45
+ box = results[0].boxes[0].xyxy[0].cpu().numpy()
46
+ return (float(box[0]), float(box[1]), float(box[2]), float(box[3]))
47
+ except Exception as e:
48
+ logger.error(f"YOLO detection failed: {e}")
49
+ return None
50
+
51
+
52
+ @functools.lru_cache(maxsize=32)
53
+ def recommend_prep_strategy(self, image_bytes: bytes) -> dict:
54
+ """
55
+ Decides whether to 'crop' or 'pad' based on object detection.
56
+ """
57
+ start_time = time.perf_counter()
58
+ image = Image.open(io.BytesIO(image_bytes))
59
+ width, height = image.size
60
+
61
+ if width == height:
62
+ return {
63
+ "strategy": PreprocessStrategy.NONE,
64
+ "reason": "Already square",
65
+ "execution_time": f"{(time.perf_counter() - start_time):.3f}s"
66
+ }
67
+
68
+ if width <= 448 and height <= 448:
69
+ return {
70
+ "strategy": PreprocessStrategy.PAD,
71
+ "reason": "Image is 448x448 or smaller; padding to square to avoid any data loss or scale-down",
72
+ "execution_time": f"{(time.perf_counter() - start_time):.3f}s"
73
+ }
74
+
75
+ bbox = self.get_lesion_bbox(image_bytes)
76
+ if not bbox:
77
+ return {
78
+ "strategy": PreprocessStrategy.CROP,
79
+ "reason": "Detection failed, defaulting to center crop",
80
+ "execution_time": f"{(time.perf_counter() - start_time):.3f}s"
81
+ }
82
+
83
+ x1, y1, x2, y2 = bbox
84
+
85
+ # Center square boundaries
86
+ new_dim = min(width, height)
87
+ if width > height:
88
+ # Landscape
89
+ crop_x1 = (width - new_dim) / 2
90
+ crop_x2 = (width + new_dim) / 2
91
+
92
+ # Check if bbox is outside the horizontal center crop
93
+ is_cut = (x1 < crop_x1) or (x2 > crop_x2)
94
+ else:
95
+ # Portrait
96
+ crop_y1 = (height - new_dim) / 2
97
+ crop_y2 = (height + new_dim) / 2
98
+
99
+ # Check if bbox is outside the vertical center crop
100
+ is_cut = (y1 < crop_y1) or (y2 > crop_y2)
101
+
102
+ if is_cut:
103
+ return {
104
+ "strategy": PreprocessStrategy.PAD,
105
+ "reason": "Object extends beyond center crop area",
106
+ "bbox": bbox,
107
+ "execution_time": f"{(time.perf_counter() - start_time):.3f}s"
108
+ }
109
+ else:
110
+ return {
111
+ "strategy": PreprocessStrategy.CROP,
112
+ "reason": "Object fully contained in center crop area",
113
+ "bbox": bbox,
114
+ "execution_time": f"{(time.perf_counter() - start_time):.3f}s"
115
+ }
116
+
117
+ def prepare_image(self, image: Image.Image, target_size: tuple = (448, 448)) -> Image.Image:
118
+ """
119
+ Intelligently prepares an image by either cropping or padding to a square,
120
+ then resizing to target_size.
121
+ """
122
+ # Convert to bytes for strategy detection
123
+ img_byte_arr = io.BytesIO()
124
+ image.save(img_byte_arr, format='JPEG')
125
+ image_bytes = img_byte_arr.getvalue()
126
+
127
+ strategy_res = self.recommend_prep_strategy(image_bytes)
128
+ strategy = strategy_res["strategy"]
129
+
130
+ width, height = image.size
131
+
132
+ if strategy == PreprocessStrategy.CROP or strategy == PreprocessStrategy.NONE:
133
+ # Traditional center crop (or already square)
134
+ new_dim = min(width, height)
135
+ left = (width - new_dim) / 2
136
+ top = (height - new_dim) / 2
137
+ right = (width + new_dim) / 2
138
+ bottom = (height + new_dim) / 2
139
+ image = image.crop((left, top, right, bottom))
140
+ elif strategy == PreprocessStrategy.PAD:
141
+ # Pad to square
142
+ new_dim = max(width, height)
143
+ # Use black background for padding as it is common for clinical vision models
144
+ new_image = Image.new("RGB", (new_dim, new_dim), (0, 0, 0))
145
+ if width > height:
146
+ # Landscape -> Pad Top/Bottom
147
+ new_image.paste(image, (0, (new_dim - height) // 2))
148
+ else:
149
+ # Portrait -> Pad Left/Right
150
+ new_image.paste(image, ((new_dim - width) // 2, 0))
151
+ image = new_image
152
+
153
+ # Finally resize
154
+ if image.size != target_size:
155
+ logger.debug(f"Resizing image to {target_size}")
156
+ image = image.resize(target_size, Image.Resampling.LANCZOS)
157
+
158
+ return image
159
+
160
+ def prepare_image_base64(self, image_bytes: bytes, target_size: tuple = (448, 448)) -> str:
161
+ """
162
+ Prepares image and returns as base64 data URI for UI debugging/display.
163
+ """
164
+ import base64
165
+ try:
166
+ image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
167
+ prepared_image = self.prepare_image(image, target_size)
168
+
169
+ buf = io.BytesIO()
170
+ prepared_image.save(buf, format="JPEG")
171
+ img_b64 = base64.b64encode(buf.getvalue()).decode('utf-8')
172
+ return f"data:image/jpeg;base64,{img_b64}"
173
+ except Exception as e:
174
+ logger.error(f"Failed to prepare image base64: {e}")
175
+ return None
176
+
177
+ def prepare_image_bytes(self, image_bytes: bytes, target_size: tuple = (448, 448)) -> bytes:
178
+ """
179
+ Helper to prepare image directly from bytes and return bytes.
180
+ """
181
+ try:
182
+ image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
183
+ prepared_image = self.prepare_image(image, target_size)
184
+
185
+ buf = io.BytesIO()
186
+ prepared_image.save(buf, format="JPEG")
187
+ return buf.getvalue()
188
+ except Exception as e:
189
+ logger.error(f"Failed to prepare image bytes: {e}")
190
+ raise e
191
+
192
+ image_preprocess_service = ImagePreprocessService()
app/services/medsiglip_modality_wrapper.py ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ from typing import List, Dict, Optional, Any
3
+ from app.services.medsiglip_service import medsiglip_service
4
+ from app.dermatology_data import MEDSIGLIP_DERMATOLOGY_NARROW_LABELS
5
+
6
+ logger = logging.getLogger(__name__)
7
+
8
+ class ClinicalModalityWrapper:
9
+ """
10
+ Generic wrapper for vision-language models that implements clinical modality templating
11
+ using the MEDSIGLIP_DERMATOLOGY_NARROW_LABELS map (Keys and Values).
12
+ """
13
+ def __init__(self, service: Any, modality: str = "macroscopic"):
14
+ self.service = service
15
+ # "macroscopic" -> "Clinical photograph showing {desc}."
16
+ # "dermoscopy" -> "Dermoscopy image revealing {desc}."
17
+ self.modality = modality
18
+ self.labels_map = MEDSIGLIP_DERMATOLOGY_NARROW_LABELS
19
+
20
+ def _get_template(self) -> str:
21
+ if self.modality == "dermoscopy":
22
+ return "Dermoscopy image revealing {}."
23
+ #return "Clinical photograph showing {}."
24
+ return "A patient-submitted smartphone photograph showing {}."
25
+
26
+ def analyze_image(self, image_bytes: bytes, custom_labels: Optional[List[str]] = None) -> List[Dict]:
27
+ """
28
+ Analyzes an image using clinical descriptions (values) wrapped in modality templates.
29
+ Returns mapped results with original short labels (keys).
30
+ """
31
+ # 1. Prepare labels and descriptions from MEDSIGLIP_DERMATOLOGY_NARROW_LABELS
32
+ if custom_labels:
33
+ descriptions = []
34
+ valid_labels = []
35
+ for label in custom_labels:
36
+ if label in self.labels_map:
37
+ descriptions.append(self.labels_map[label])
38
+ valid_labels.append(label)
39
+ else:
40
+ # If not in our clinical map, use original label as description
41
+ descriptions.append(label)
42
+ valid_labels.append(label)
43
+ else:
44
+ # Use all predefined clinical labels (Keys and Values)
45
+ valid_labels = list(self.labels_map.keys())
46
+ descriptions = list(self.labels_map.values())
47
+
48
+ # 2. Apply modality template to descriptions (Values)
49
+ template = self._get_template()
50
+ prompts = [template.format(desc) for desc in descriptions]
51
+
52
+ print(f"\n[DEBUG] Prompts for {self.service.model_name}:")
53
+ for p in prompts:
54
+ print(f" - {p}")
55
+
56
+ # 3. Call the underlying service
57
+ # Handle different method names between MedSigLIP and SigLIP services
58
+ if hasattr(self.service, "get_embeddings"):
59
+ raw_results = self.service.get_embeddings(image_bytes, texts=prompts)
60
+ elif hasattr(self.service, "get_predictions"):
61
+ raw_results = self.service.get_predictions(image_bytes, texts=prompts)
62
+ else:
63
+ raise AttributeError(f"Service {type(self.service)} has no supported inference method.")
64
+
65
+ # 4. Map prompts back to original short labels (Keys)
66
+ prompt_to_label = dict(zip(prompts, valid_labels))
67
+
68
+ mapped_results = []
69
+ for res in raw_results:
70
+ original_label = prompt_to_label.get(res["label"], res["label"])
71
+ mapped_results.append({
72
+ "label": original_label,
73
+ "description": res["label"], # The full prompt used
74
+ "score": res["score"]
75
+ })
76
+
77
+ return mapped_results
78
+
79
+ # Global instances for easy access
80
+ medsiglip_wrapped_service = ClinicalModalityWrapper(medsiglip_service)
app/services/medsiglip_service.py ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import torch
3
+ import os
4
+ from PIL import Image
5
+ from transformers import AutoProcessor, AutoModel
6
+ import io
7
+ from typing import List, Optional
8
+ from app.services.image_preprocess_service import image_preprocess_service
9
+ from app.config import MEDSIGLIP_MODEL_NAME, MODEL_IMAGE_SIZE
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+ class MedSigLIPService:
14
+ def __init__(self, model_name=MEDSIGLIP_MODEL_NAME):
15
+ # We'll lazy load the model to avoid startup costs and potential auth issues crashing the app immediately
16
+ self.model_name = model_name
17
+
18
+ self.processor = None
19
+ self.model = None
20
+ if torch.cuda.is_available():
21
+ self.device = "cuda"
22
+ elif torch.backends.mps.is_available():
23
+ self.device = "mps"
24
+ else:
25
+ self.device = "cpu"
26
+
27
+ def _load_model(self):
28
+ if self.model is None:
29
+ logger.info(f"Loading MedSigLIP model: {self.model_name} on {self.device}...")
30
+ try:
31
+ token = os.getenv("HF_TOKEN")
32
+ self.processor = AutoProcessor.from_pretrained(self.model_name, token=token)
33
+ self.model = AutoModel.from_pretrained(self.model_name, token=token).to(self.device)
34
+ logger.info("MedSigLIP model loaded successfully.")
35
+ except Exception as e:
36
+ logger.error(f"Failed to load MedSigLIP model: {e}")
37
+ raise e
38
+
39
+ def get_embeddings(self, image_bytes: bytes, texts: Optional[List[str]] = None):
40
+ """
41
+ Run inference to get embeddings or probabilities for zero-shot classification.
42
+ If texts is provided, performs zero-shot classification via similarity.
43
+ """
44
+ self._load_model()
45
+ try:
46
+ image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
47
+ image = image_preprocess_service.prepare_image(image, MODEL_IMAGE_SIZE)
48
+
49
+ if texts:
50
+ # 64-token limit check as per requirements
51
+ inputs = self.processor(
52
+ text=texts,
53
+ images=image,
54
+ padding="max_length",
55
+ max_length=64,
56
+ truncation=True,
57
+ return_tensors="pt"
58
+ ).to(self.device)
59
+
60
+ # Optional: Log warning if truncation occurred (check input_ids shape vs max_length)
61
+ # Note: with truncation=True, the shape will be (num_texts, 64)
62
+ # To detect if it *would* have exceeded, we could tokenize without truncation first,
63
+ # but that's expensive. Instead, we can just ensure we stay within the limit.
64
+
65
+ with torch.no_grad():
66
+ outputs = self.model(**inputs)
67
+
68
+ # Retrieve logits
69
+ logits_per_image = outputs.logits_per_image
70
+ probs = logits_per_image.softmax(dim=1)
71
+
72
+ # Format results
73
+ results = []
74
+ prob_values = probs[0].tolist()
75
+ for i, text in enumerate(texts):
76
+ results.append({"label": text, "score": prob_values[i]})
77
+
78
+ # Sort by score descending
79
+ results.sort(key=lambda x: x["score"], reverse=True)
80
+ return results
81
+ else:
82
+ # Just image embedding
83
+ # MedSigLIP is a CLIP-like model, so we can get features
84
+ inputs = self.processor(images=image, return_tensors="pt").to(self.device) # Only image
85
+ with torch.no_grad():
86
+ image_features = self.model.get_image_features(**inputs)
87
+
88
+ return {"embedding": image_features[0].tolist()}
89
+
90
+ except Exception as e:
91
+ logger.error(f"MedSigLIP inference failed: {e}")
92
+ raise e
93
+
94
+ # Global instance
95
+ medsiglip_service = MedSigLIPService()
app/services/result_interpreter.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import math
2
+ import logging
3
+ from typing import List, Dict, Any
4
+ from app.dermatology_data import CANCEROUS_TUMOR_CLASSES
5
+ from app.config import INTERPRETER_ENTROPY_THRESHOLD, INTERPRETER_MARGIN_THRESHOLD, CONFIDENCE_CLASSES
6
+
7
+ logger = logging.getLogger(__name__)
8
+
9
+ class ResultInterpreter:
10
+ """
11
+ Analyzes classification results from MedSigLIP models to provide clinical insights.
12
+
13
+ Responsibilities:
14
+ 1. Detect if top predictions indicate tumor-related diseases based on CANCEROUS_TUMOR_CLASSES.
15
+ 2. Handle mixed cases (Tumor vs Non-Tumor) with confidence margins.
16
+ 3. Calculate Predictive Entropy (Shannon Entropy) as a measure of model uncertainty.
17
+ 4. Provide descriptive annotations and color hints for UI.
18
+ 5. Classify confidence based on Top-1 vs Top-2 margin.
19
+ """
20
+
21
+ def interpret(self, results: List[Dict[str, Any]],
22
+ entropy_threshold: float = INTERPRETER_ENTROPY_THRESHOLD,
23
+ margin_threshold: float = INTERPRETER_MARGIN_THRESHOLD) -> Dict[str, Any]:
24
+ """Interprets a list of classification results."""
25
+ if not results:
26
+ return self._empty_result()
27
+
28
+ scores = [r["score"] for r in results]
29
+ entropy = self.calculate_entropy(scores)
30
+ is_reliable = entropy < entropy_threshold
31
+
32
+ # Rule 1: Margin Calculation (including Tumor Consolidation)
33
+ margin = self._calculate_margin(results)
34
+ conf_info = self.get_confidence_level(margin)
35
+
36
+ # Rule 2: Status and Annotation Logic
37
+ analysis = self._determine_status_and_annotation(results, margin, margin_threshold)
38
+
39
+ # Rule 3: Format computation process for tech logs
40
+ comp_process = self._format_computation_process(
41
+ results, margin, margin_threshold, conf_info, entropy, entropy_threshold, is_reliable
42
+ )
43
+
44
+ return {
45
+ "is_high_risk": analysis["is_high_risk"],
46
+ "entropy": entropy,
47
+ "is_reliable": is_reliable,
48
+ "annotation": analysis["annotation"],
49
+ "color_hint": analysis["color_hint"],
50
+ "confidence_label": conf_info["label"],
51
+ "confidence_color": conf_info["color_hint"],
52
+ "status": analysis["status"],
53
+ "margin": margin,
54
+ "margin_threshold": margin_threshold,
55
+ "computation_process": comp_process,
56
+ "top_2_labels": [results[0]["label"], results[1]["label"]] if len(results) > 1 else [results[0]["label"], "None"]
57
+ }
58
+
59
+ def _empty_result(self) -> Dict[str, Any]:
60
+ return {
61
+ "is_high_risk": False, "entropy": 0.0, "is_reliable": False,
62
+ "annotation": "No results available to interpret.", "color_hint": "gray",
63
+ "confidence_label": "Unknown", "confidence_color": "gray",
64
+ "computation_process": ["No results provided."]
65
+ }
66
+
67
+ def _calculate_margin(self, results: List[Dict[str, Any]]) -> float:
68
+ """
69
+ Calculates margin.
70
+ Tumor rule: sum(contiguous tumors) - first_non_tumor
71
+ Default rule: top_1 - top_2
72
+ """
73
+ top_1 = results[0]
74
+ if top_1["label"] in CANCEROUS_TUMOR_CLASSES:
75
+ tumor_sum = 0.0
76
+ next_non_tumor_score = 0.0
77
+ found_non_tumor = False
78
+ for r in results:
79
+ if not found_non_tumor and r["label"] in CANCEROUS_TUMOR_CLASSES:
80
+ tumor_sum += r["score"]
81
+ elif not found_non_tumor:
82
+ next_non_tumor_score = r["score"]
83
+ found_non_tumor = True
84
+ return round(tumor_sum - next_non_tumor_score, 4)
85
+
86
+ top_2_score = results[1]["score"] if len(results) > 1 else 0.0
87
+ return round(top_1["score"] - top_2_score, 4)
88
+
89
+ def _determine_status_and_annotation(self, results: List[Dict[str, Any]], margin: float, margin_threshold: float) -> Dict[str, Any]:
90
+ """Provides status, annotation, and color hint based on top results."""
91
+ t1 = results[0]
92
+ t2 = results[1] if len(results) > 1 else {"label": "None", "score": 0.0}
93
+
94
+ is_t1_tumor = t1["label"] in CANCEROUS_TUMOR_CLASSES
95
+ is_t2_tumor = t2["label"] in CANCEROUS_TUMOR_CLASSES
96
+
97
+ if is_t1_tumor and is_t2_tumor:
98
+ return {"is_high_risk": True, "annotation": "High likeness of tumor disease", "color_hint": "red", "status": "tumor_detected"}
99
+
100
+ if is_t1_tumor != is_t2_tumor:
101
+ if margin < margin_threshold:
102
+ return {"is_high_risk": False, "annotation": "Not clear", "color_hint": "yellow", "status": "uncertain_mixed"}
103
+
104
+ if is_t1_tumor:
105
+ return {"is_high_risk": False, "annotation": f"Potential cancerous condition: {t1['label']}", "color_hint": "red", "status": "potential_tumor"}
106
+ return {"is_high_risk": False, "annotation": f"Likely benign: {t1['label']}", "color_hint": "green", "status": "likely_benign"}
107
+
108
+ return {"is_high_risk": False, "annotation": "No immediate tumor likeness detected in top results", "color_hint": "green", "status": "benign"}
109
+
110
+ def _format_computation_process(self, results, margin, margin_threshold, conf_info, entropy, entropy_threshold, is_reliable) -> List[str]:
111
+ """Formats the detailed steps of interpretation for UI display."""
112
+ t1 = results[0]
113
+ t2 = results[1] if len(results) > 1 else {"label": "None", "score": 0.0}
114
+
115
+ process = [
116
+ f"Top 1: {t1['label']} ({t1['score']:.2f}) - Tumor: {t1['label'] in CANCEROUS_TUMOR_CLASSES}",
117
+ f"Top 2: {t2['label']} ({t2['score']:.2f}) - Tumor: {t2['label'] in CANCEROUS_TUMOR_CLASSES}",
118
+ f"Margin: {margin:.4f} (Threshold: {margin_threshold})",
119
+ f"Confidence: {conf_info['label']}",
120
+ f"Mixed Case Detect: {'Yes' if (t1['label'] in CANCEROUS_TUMOR_CLASSES) != (t2['label'] in CANCEROUS_TUMOR_CLASSES) else 'No'}",
121
+ f"Entropy: {entropy:.2f} bits (Limit: {entropy_threshold})"
122
+ ]
123
+ process.append("Status: Prediction within reliability limits" if is_reliable else f"Status: Low confidence - High uncertainty detected (Entropy: {entropy:.2f})")
124
+ return process
125
+
126
+ def get_confidence_level(self, margin: float) -> Dict[str, str]:
127
+ """
128
+ Maps margin to a qualitative confidence level using thresholds from config.
129
+ """
130
+ # Round to avoid floating point precision issues (e.g. 0.4 - 0.1 = 0.30000000000000004)
131
+ m = round(margin, 4)
132
+
133
+ for cls in CONFIDENCE_CLASSES:
134
+ if m > cls["min"]:
135
+ return {
136
+ "label": cls["label"],
137
+ "color_hint": cls.get("color_hint", "")
138
+ }
139
+
140
+ # Fallback to the last class (usually 0.0) if no match found
141
+ last_cls = CONFIDENCE_CLASSES[-1]
142
+ return {
143
+ "label": last_cls["label"],
144
+ "color_hint": last_cls.get("color_hint", "")
145
+ }
146
+
147
+ def calculate_entropy(self, probabilities: List[float]) -> float:
148
+ """
149
+ Calculates Shannon entropy in bits.
150
+ H = -sum(pi * log2(pi))
151
+ """
152
+ entropy = 0.0
153
+ for p in probabilities:
154
+ if p > 1e-9: # Avoid log(0)
155
+ entropy -= p * math.log2(p)
156
+ return entropy
157
+
158
+ # Global singleton instance
159
+ result_interpreter = ResultInterpreter()
app/services/vertex_client.py ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import logging
3
+ try:
4
+ from google.cloud import aiplatform
5
+ from google.oauth2 import service_account
6
+ import google.auth
7
+ GOOGLE_CLOUD_AVAILABLE = True
8
+ except ImportError:
9
+ GOOGLE_CLOUD_AVAILABLE = False
10
+ aiplatform = None
11
+ service_account = None
12
+ google = None
13
+
14
+ logger = logging.getLogger(__name__)
15
+
16
+ class VertexClient:
17
+ def __init__(self):
18
+ self.project_id = os.environ.get("PROJECT_ID")
19
+ self.location = os.environ.get("LOCATION", "us-central1")
20
+ self.endpoint_id = os.environ.get("ENDPOINT_ID") # ID of the deployed MedGemma endpoint
21
+ self.credentials_path = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")
22
+
23
+ self.setup_complete = False
24
+
25
+ if not GOOGLE_CLOUD_AVAILABLE:
26
+ logger.warning("google-cloud-aiplatform not installed. Vertex AI client will be mocked.")
27
+ return
28
+
29
+ if self.project_id:
30
+ try:
31
+ # If credentials path is set, explicit load (dev), else default (cloud run)
32
+ if self.credentials_path and os.path.exists(self.credentials_path):
33
+ creds = service_account.Credentials.from_service_account_file(self.credentials_path)
34
+ else:
35
+ creds, _ = google.auth.default()
36
+
37
+ aiplatform.init(
38
+ project=self.project_id,
39
+ location=self.location,
40
+ credentials=creds
41
+ )
42
+ self.setup_complete = True
43
+ logger.info(f"Vertex AI initialized for project {self.project_id}")
44
+ except Exception as e:
45
+ logger.error(f"Failed to initialize Vertex AI: {e}")
46
+
47
+ async def predict(self, prompt: str, max_tokens: int = 256, temperature: float = 0.2) -> str:
48
+ if not self.setup_complete:
49
+ logger.info("Returning mock prediction because Vertex AI is not configured.")
50
+ return "Mock Response: Vertex AI is not configured. This is a dummy prediction."
51
+
52
+ if not self.endpoint_id:
53
+ return "Endpoint ID not configured."
54
+
55
+ try:
56
+ # Get Endpoint
57
+ endpoint = aiplatform.Endpoint(self.endpoint_id)
58
+
59
+ # Predict
60
+ # Structure depends on the model serving container.
61
+ # MedGemma usually expects instances=[{"prompt": ...}]
62
+ instances = [{"prompt": prompt, "max_tokens": max_tokens, "temperature": temperature}]
63
+
64
+ response = endpoint.predict(instances=instances)
65
+
66
+ # Parse prediction (assuming standard format, adjust based on actual model output)
67
+ # Typically response.predictions is a list
68
+ if response.predictions:
69
+ return str(response.predictions[0])
70
+ else:
71
+ return "No prediction returned."
72
+
73
+ except Exception as e:
74
+ logger.error(f"Prediction failed: {e}")
75
+ raise e
76
+
77
+ vertex_client = VertexClient()
app/services/yolo_service.py ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import os
3
+
4
+ logger = logging.getLogger(__name__)
5
+
6
+ class YOLOService:
7
+ def __init__(self):
8
+ self.model = None
9
+
10
+ def load_model(self):
11
+ if self.model is None:
12
+ try:
13
+ from ultralytics import YOLO
14
+ # Use YOLOv8-Nano
15
+ logger.info("Loading YOLOv8-Nano model...")
16
+ self.model = YOLO('yolov8n.pt')
17
+ except ImportError:
18
+ logger.warning("ultralytics not installed. YOLO detection will be skipped.")
19
+ return None
20
+ return self.model
21
+
22
+ yolo_service = YOLOService()
app/static/app.js ADDED
@@ -0,0 +1,512 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ function dermatologApp() {
2
+ return {
3
+ // App State
4
+ activeTab: 'photos',
5
+ analysisResults: {},
6
+ showTechnicalDetails: {}, // Map of photo_id -> boolean
7
+
8
+ // Chat State
9
+ prompt: '',
10
+ temperature: 0.2,
11
+ loading: false,
12
+ response: null,
13
+ latency: null,
14
+ sessionId: null,
15
+
16
+ // Photo State
17
+ timeline: [],
18
+ dragover: false,
19
+ editingPhoto: null,
20
+ editingDate: '',
21
+
22
+ // Common
23
+ error: null,
24
+ modelName: 'Loading...',
25
+ yoloAvailable: false,
26
+ marginThreshold: 0.05,
27
+ currentAnalysisId: null,
28
+ clearPromise: null,
29
+ debugMode: false,
30
+
31
+ init() {
32
+ this.sessionId = this.getCookie('session_id');
33
+ const urlParams = new URLSearchParams(window.location.search);
34
+ this.debugMode = urlParams.has('debug');
35
+
36
+ this.loadTimeline();
37
+ this.fetchModelInfo();
38
+
39
+ // Global Paste Handler
40
+ window.addEventListener('paste', (e) => {
41
+ const items = (e.clipboardData || e.originalEvent.clipboardData).items;
42
+ const files = [];
43
+ for (let i = 0; i < items.length; i++) {
44
+ if (items[i].type.indexOf('image') !== -1) {
45
+ const file = items[i].getAsFile();
46
+ if (file) files.push(file);
47
+ }
48
+ }
49
+ if (files.length > 0) {
50
+ this.handleFiles(files);
51
+ }
52
+ });
53
+
54
+ // Prevent accidental refresh
55
+ window.addEventListener('beforeunload', (e) => {
56
+ if (this.timeline && this.timeline.length > 0) {
57
+ const msg = "On refresh the content would be cleared. Are you sure you want to leave?";
58
+ e.preventDefault();
59
+ e.returnValue = msg;
60
+ return msg;
61
+ }
62
+ });
63
+ },
64
+
65
+ async fetchModelInfo() {
66
+ try {
67
+ const res = await fetch('/api/health');
68
+ if (res.ok) {
69
+ const data = await res.json();
70
+ this.yoloAvailable = data.yolo_available;
71
+ if (data.status === "OK") {
72
+ this.modelName = "MedSigLIP (Local)";
73
+ } else if (data.status === "suspended") {
74
+ this.modelName = "Service Suspended";
75
+ } else {
76
+ this.modelName = data.status || "Unknown Status";
77
+ }
78
+ }
79
+ } catch (e) {
80
+ console.error("Failed to fetch model info", e);
81
+ this.modelName = "Error fetching health";
82
+ }
83
+ },
84
+
85
+ getCookie(name) {
86
+ const value = `; ${document.cookie}`;
87
+ const parts = value.split(`; ${name}=`);
88
+ if (parts.length === 2) return parts.pop().split(';').shift();
89
+ return null;
90
+ },
91
+
92
+ async loadTimeline() {
93
+ try {
94
+ const res = await fetch('/api/photos?t=' + new Date().getTime());
95
+ if (res.ok) {
96
+ this.timeline = await res.json();
97
+
98
+ this.timeline.forEach(item => {
99
+ const processPhoto = (p) => {
100
+ // Status is handled at runtime in this.analysisResults, not restored from DB
101
+ // satisfy "store the state whether image was processed in the html not db"
102
+ };
103
+
104
+ if (item.type === 'directory') {
105
+ item.items.forEach(processPhoto);
106
+ } else if (item.type === 'photo') {
107
+ processPhoto(item.data);
108
+ }
109
+ });
110
+ }
111
+ } catch (e) {
112
+ console.error("Timeline load failed", e);
113
+ }
114
+ },
115
+
116
+ async handleDrop(event) {
117
+ this.dragover = false;
118
+ const files = event.dataTransfer.files;
119
+ if (files.length > 0) {
120
+ this.handleFiles(files);
121
+ }
122
+ },
123
+
124
+ async handleFiles(files) {
125
+ if (files.length === 0) return;
126
+
127
+ // Ensure we wait for any ongoing session clearing to finish
128
+ if (this.clearPromise) {
129
+ await this.clearPromise;
130
+ }
131
+
132
+ this.loading = true;
133
+ try {
134
+ for (let i = 0; i < files.length; i++) {
135
+ const file = files[i];
136
+ const dataUrl = await this.readAsDataURL(file);
137
+
138
+ // Basic duplicate check (by name and size for local)
139
+ const isDuplicate = this.getAllPhotos().some(p => p.filename === file.name && p.size === file.size);
140
+ if (isDuplicate) {
141
+ this.showToast("Upload Notice", `Skipped ${file.name} (already in timeline)`, "warning");
142
+ continue;
143
+ }
144
+
145
+ const photoId = crypto.randomUUID();
146
+ const photo = {
147
+ id: photoId,
148
+ filename: file.name,
149
+ size: file.size,
150
+ creation_date: new Date(file.lastModified || Date.now()).toISOString().split('T')[0],
151
+ uploaded_at: new Date().toISOString(),
152
+ local_content: dataUrl,
153
+ analysis: null
154
+ };
155
+
156
+ this.addPhotoToTimeline(photo);
157
+ }
158
+
159
+ // Brief delay to let UI render the new cards
160
+ setTimeout(() => {
161
+ this.analyzeAllPhotos();
162
+ }, 300);
163
+
164
+ } catch (e) {
165
+ console.error("Local processing error:", e);
166
+ this.error = "Failed to process images: " + e.message;
167
+ } finally {
168
+ this.loading = false;
169
+ }
170
+ },
171
+
172
+ readAsDataURL(file) {
173
+ return new Promise((resolve, reject) => {
174
+ const reader = new FileReader();
175
+ reader.onload = () => resolve(reader.result);
176
+ reader.onerror = reject;
177
+ reader.readAsDataURL(file);
178
+ });
179
+ },
180
+
181
+ addPhotoToTimeline(photo) {
182
+ // Check if day exists
183
+ let dir = this.timeline.find(item => item.type === 'directory' && item.date === photo.creation_date);
184
+ if (!dir) {
185
+ dir = {
186
+ type: 'directory',
187
+ date: photo.creation_date,
188
+ items: [],
189
+ count: 0
190
+ };
191
+ this.timeline.push(dir);
192
+ // Sort timeline by date descending
193
+ this.timeline.sort((a, b) => b.date.localeCompare(a.date));
194
+ }
195
+
196
+ // Avoid duplicates in items list
197
+ if (!dir.items.some(p => p.id === photo.id)) {
198
+ dir.items.push(photo);
199
+ dir.items.sort((a, b) => b.uploaded_at.localeCompare(a.uploaded_at));
200
+ dir.count = dir.items.length;
201
+ }
202
+ },
203
+
204
+ async deletePhoto(photoId) {
205
+ if (!confirm("Are you sure you want to delete this photo locally?")) return;
206
+
207
+ // Remove from timeline state (purely local)
208
+ this.timeline.forEach(dir => {
209
+ if (dir.type === 'directory') {
210
+ dir.items = dir.items.filter(p => p.id !== photoId);
211
+ dir.count = dir.items.length;
212
+ }
213
+ });
214
+ // Clean up empty directories
215
+ this.timeline = this.timeline.filter(dir => dir.type !== 'directory' || dir.count > 0);
216
+
217
+ delete this.analysisResults[photoId];
218
+ return true;
219
+ },
220
+
221
+ async deletePhotoFromModal() {
222
+ if (!this.editingPhoto) return;
223
+ const success = await this.deletePhoto(this.editingPhoto.id);
224
+ if (success) {
225
+ document.querySelector('.edit-dialog').hide();
226
+ this.editingPhoto = null;
227
+ }
228
+ },
229
+
230
+ async clearSession() {
231
+ if (!confirm("Clear all local photos?")) return;
232
+ // reset all frontend reactive state variables needed for a clean run
233
+ this.analysisResults = {};
234
+ this.timeline = [];
235
+ this.showTechnicalDetails = {};
236
+ this.prompt = '';
237
+ this.loading = false;
238
+ this.response = null;
239
+ this.latency = null;
240
+ this.currentAnalysisId = null;
241
+ this.editingPhoto = null;
242
+ this.editingDate = '';
243
+
244
+ // Reset file inputs so identical files can trigger @change again
245
+ if (this.$refs.fileInput) this.$refs.fileInput.value = '';
246
+ if (this.$refs.cameraInput) this.$refs.cameraInput.value = '';
247
+
248
+ // Optional: Tell backend to clear its session context if needed
249
+ fetch('/api/photos', { method: 'DELETE' }).catch(console.error);
250
+ },
251
+
252
+
253
+ openEditModal(photo) {
254
+ this.editingPhoto = photo;
255
+ this.editingDate = photo.creation_date;
256
+ document.querySelector('.edit-dialog').show();
257
+ },
258
+
259
+ async saveDate() {
260
+ if (!this.editingPhoto) return;
261
+
262
+ // Update locally
263
+ const oldDate = this.editingPhoto.creation_date;
264
+ const newDate = this.editingDate;
265
+
266
+ if (oldDate !== newDate) {
267
+ // Remove from old location
268
+ this.timeline.forEach(dir => {
269
+ if (dir.date === oldDate) {
270
+ dir.items = dir.items.filter(p => p.id !== this.editingPhoto.id);
271
+ dir.count = dir.items.length;
272
+ }
273
+ });
274
+
275
+ // Add to new
276
+ this.editingPhoto.creation_date = newDate;
277
+ this.addPhotoToTimeline(this.editingPhoto);
278
+
279
+ // Cleanup empty
280
+ this.timeline = this.timeline.filter(dir => dir.count > 0);
281
+ }
282
+
283
+ document.querySelector('.edit-dialog').hide();
284
+ this.editingPhoto = null;
285
+ },
286
+
287
+ async analyzeAllPhotos() {
288
+ this.loading = true;
289
+ this.error = null;
290
+ this.latency = 0;
291
+
292
+ const photos = this.getAllPhotos();
293
+ if (photos.length === 0) {
294
+ this.loading = false;
295
+ return;
296
+ }
297
+
298
+ const photosToAnalyze = photos.filter(p => !this.analysisResults[p.id]);
299
+ if (photosToAnalyze.length === 0) {
300
+ this.loading = false;
301
+ return;
302
+ }
303
+
304
+ let report = (this.response || "");
305
+ if (report && !report.endsWith("\n\n")) report += "\n\n";
306
+ report += `--- Starting Local Analysis Batch [${new Date().toLocaleTimeString()}] ---\n`;
307
+ this.response = report;
308
+
309
+ let startTime = performance.now();
310
+
311
+ try {
312
+ for (const photo of photosToAnalyze) {
313
+ console.log(`Starting analysis for ${photo.filename} (${photo.id})`);
314
+ report += `Analyzing ${photo.filename} (Local Transfer)...\n`;
315
+ this.response = report;
316
+ this.currentAnalysisId = photo.id;
317
+
318
+ try {
319
+ const res = await fetch(`/api/photos/${photo.id}/analyze`, {
320
+ method: 'POST',
321
+ headers: { 'Content-Type': 'application/json' },
322
+ body: JSON.stringify({
323
+ model: 'medsiglip',
324
+ margin_threshold: parseFloat(this.marginThreshold),
325
+ base64_image: photo.local_content
326
+ })
327
+ });
328
+
329
+ if (res.ok) {
330
+ const data = await res.json();
331
+ if (data.predictions && data.predictions.length > 0) {
332
+ // Populate results for UI
333
+ this.analysisResults[photo.id] = {
334
+ id: photo.id,
335
+ date: new Date().toISOString(),
336
+ prediction: data.predictions[0],
337
+ predictions: data.predictions, // For legacy if any
338
+ primary: data.predictions,
339
+ initial_classification: data.initial_classification,
340
+ primary_name: data.primary_model_name,
341
+ interpretation: data.interpretation,
342
+ preprocess_strategy: data.preprocess_strategy,
343
+ prepared_image_base64: data.prepared_image_base64,
344
+ execution_times: data.execution_times,
345
+ saliency_base64: data.saliency_base64
346
+ };
347
+ report += ` ➔ Primary Results (${data.primary_model_name}):\n`;
348
+ data.predictions.forEach(p => {
349
+ report += ` - ${p.label}: ${(p.score * 100).toFixed(1)}%\n`;
350
+ });
351
+ }
352
+ } else {
353
+ const err = await res.text();
354
+ report += ` ➔ Request Failed: ${res.status} ${err}\n`;
355
+ }
356
+ } catch (e) {
357
+ console.error(`Analysis error for ${photo.id}:`, e);
358
+ report += ` ➔ Error: ${e.message}\n`;
359
+ }
360
+
361
+ this.currentAnalysisId = null;
362
+ report += "\n";
363
+ this.response = report;
364
+ }
365
+
366
+ this.latency = Math.round(performance.now() - startTime);
367
+ report += "Batch Completion Success.";
368
+ this.response = report;
369
+
370
+ } catch (e) {
371
+ console.error(e);
372
+ this.error = "Analysis process encountered a critical error.";
373
+ } finally {
374
+ this.loading = false;
375
+ }
376
+ },
377
+
378
+ async fetchSaliency(photo) {
379
+ console.log("fetchSaliency triggered for", photo.id);
380
+ if (!this.analysisResults[photo.id]) {
381
+ console.warn("No analysis results for photo", photo.id);
382
+ return;
383
+ }
384
+ if (this.analysisResults[photo.id].saliency_base64) {
385
+ console.log("Saliency already exists for", photo.id);
386
+ return;
387
+ }
388
+ if (!this.analysisResults[photo.id].primary || this.analysisResults[photo.id].primary.length === 0) {
389
+ console.warn("No primary assessment predictions for", photo.id);
390
+ return;
391
+ }
392
+
393
+ const topLabel = this.analysisResults[photo.id].primary[0].label;
394
+ console.log("Fetching saliency for label:", topLabel);
395
+
396
+ try {
397
+ const res = await fetch(`/api/photos/${photo.id}/saliency`, {
398
+ method: 'POST',
399
+ headers: { 'Content-Type': 'application/json' },
400
+ body: JSON.stringify({
401
+ base64_image: photo.local_content,
402
+ target_label: topLabel
403
+ })
404
+ });
405
+
406
+ if (res.ok) {
407
+ const data = await res.json();
408
+ console.log("Saliency data received for", photo.id, "len:", data.saliency_base64 ? data.saliency_base64.length : 0);
409
+ this.analysisResults[photo.id].saliency_base64 = data.saliency_base64;
410
+ console.log("Updated analysisResults with saliency for", photo.id);
411
+ } else {
412
+ console.error("Saliency fetch failed with status:", res.status);
413
+ }
414
+ } catch (e) {
415
+ console.error("Saliency fetch error:", e);
416
+ }
417
+ },
418
+
419
+ getAllPhotos() {
420
+ let photos = [];
421
+ this.timeline.forEach(item => {
422
+ if (item.type === 'photo') photos.push(item.data);
423
+ else if (item.type === 'directory') photos.push(...item.items);
424
+ });
425
+ return photos;
426
+ },
427
+
428
+ getInterpretationColor(hint) {
429
+ const colors = {
430
+ 'red': 'var(--sl-color-danger-600)',
431
+ 'yellow': 'var(--sl-color-warning-600)',
432
+ 'green': 'var(--sl-color-success-600)',
433
+ 'gray': 'var(--sl-color-neutral-600)'
434
+ };
435
+ return colors[hint] || colors['gray'];
436
+ },
437
+
438
+ getBadgeVariant(hint) {
439
+ const variants = {
440
+ 'green': 'success',
441
+ 'gray': 'neutral',
442
+ 'yellow': 'warning',
443
+ 'red': 'danger'
444
+ };
445
+ return variants[hint] || 'neutral';
446
+ },
447
+
448
+ getInterpretationIcon(hint) {
449
+ if (hint === 'green') return 'shield-check';
450
+ if (hint === 'red') return 'exclamation-triangle';
451
+ return 'activity';
452
+ },
453
+
454
+ copyReport(photoId) {
455
+ const result = this.analysisResults[photoId];
456
+ if (!result) return;
457
+
458
+ const annotation = result.interpretation ? result.interpretation.annotation : result.prediction.label;
459
+ const confidence = result.interpretation ? result.interpretation.confidence_label : 'N/A';
460
+ const score = Math.round(result.prediction.score * 100) + '%';
461
+
462
+ const text = `Clinical Summary\n----------------\nResult: ${annotation}\nConfidence: ${confidence} (${score})\nDate: ${new Date(result.date).toLocaleString()}\n\nNote: This is an AI-assisted analysis and should be reviewed by a professional.`;
463
+
464
+ navigator.clipboard.writeText(text).then(() => {
465
+ this.showToast('Copied', 'Clinical summary copied to clipboard', 'success', 'clipboard-check');
466
+ });
467
+ },
468
+
469
+ toggleDebug() {
470
+ this.debugMode = !this.debugMode;
471
+ const url = new URL(window.location.href);
472
+ if (this.debugMode) {
473
+ url.searchParams.set('debug', '1');
474
+ } else {
475
+ url.searchParams.delete('debug');
476
+ }
477
+ window.history.replaceState({}, '', url.toString());
478
+ },
479
+
480
+ showToast(title, message, variant = 'primary', icon = 'info-circle') {
481
+ const alert = Object.assign(document.createElement('sl-alert'), {
482
+ variant: variant,
483
+ closable: true,
484
+ duration: 5000,
485
+ innerHTML: `
486
+ <sl-icon slot="icon" name="${icon}"></sl-icon>
487
+ <strong>${title}</strong><br />
488
+ ${message}
489
+ `
490
+ });
491
+ document.body.append(alert);
492
+
493
+ // Ensure shoelace components are defined before calling methods
494
+ if (typeof customElements !== 'undefined' && customElements.whenDefined) {
495
+ customElements.whenDefined('sl-alert').then(() => {
496
+ if (typeof alert.toast === 'function') {
497
+ alert.toast();
498
+ }
499
+ });
500
+ } else {
501
+ // Fallback for environments where customElements/Shoelace might not be fully loaded
502
+ setTimeout(() => {
503
+ if (typeof alert.toast === 'function') alert.toast();
504
+ }, 100);
505
+ }
506
+ }
507
+ }
508
+ }
509
+
510
+ if (typeof window !== 'undefined') {
511
+ window.dermatologApp = dermatologApp;
512
+ }
app/static/img/body_outline.svg ADDED
app/static/js/modules/api.js ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * API client for backend communication
3
+ */
4
+ export class ApiClient {
5
+ /**
6
+ * @param {string} baseUrl - Base URL for API (default: current origin)
7
+ */
8
+ constructor(baseUrl = '') {
9
+ this.baseUrl = baseUrl;
10
+ }
11
+
12
+ /**
13
+ * Uploads an image for analysis
14
+ * @param {FormData} formData - Form data containing image file
15
+ * @returns {Promise<Object>} { task_id: string }
16
+ * @throws {Error} If upload fails
17
+ */
18
+ async uploadImage(formData) {
19
+ const response = await fetch(`${this.baseUrl}/upload`, {
20
+ method: 'POST',
21
+ body: formData
22
+ });
23
+
24
+ if (!response.ok) {
25
+ throw new Error(`Upload failed: ${response.statusText}`);
26
+ }
27
+
28
+ return response.json();
29
+ }
30
+
31
+ /**
32
+ * Gets progress for a task
33
+ * @param {string} taskId - Task ID
34
+ * @returns {Promise<Object|null>} Task data or null if not found
35
+ */
36
+ async getProgress(taskId) {
37
+ try {
38
+ const response = await fetch(`${this.baseUrl}/progress/${taskId}`);
39
+ if (!response.ok) return null;
40
+ return response.json();
41
+ } catch (error) {
42
+ console.error('Failed to fetch progress:', error);
43
+ return null;
44
+ }
45
+ }
46
+
47
+ /**
48
+ * Gets aggregate statistics
49
+ * @returns {Promise<Object>} Statistics data
50
+ * @throws {Error} If fetch fails
51
+ */
52
+ async getStats() {
53
+ const response = await fetch(`${this.baseUrl}/stats`);
54
+ if (!response.ok) {
55
+ throw new Error(`Failed to fetch stats: ${response.statusText}`);
56
+ }
57
+ return response.json();
58
+ }
59
+ }
app/templates/index.html ADDED
@@ -0,0 +1,1149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+
4
+ <head>
5
+ <meta charset="UTF-8">
6
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
7
+ <title>Dermatolog AI Scan</title>
8
+
9
+ <!-- Custom JS (Must load before Alpine) -->
10
+ <script defer src="/static/app.js?v=22"></script>
11
+
12
+ <!-- Alpine.js -->
13
+ <script defer src="https://cdn.jsdelivr.net/npm/alpinejs@3.13.3/dist/cdn.min.js"></script>
14
+
15
+ <!-- Shoelace -->
16
+ <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@shoelace-style/shoelace@2.12.0/cdn/themes/light.css" />
17
+ <script type="module"
18
+ src="https://cdn.jsdelivr.net/npm/@shoelace-style/shoelace@2.12.0/cdn/shoelace-autoloader.js"></script>
19
+
20
+
21
+
22
+ <!-- Fonts -->
23
+ <link rel="preconnect" href="https://fonts.googleapis.com">
24
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
25
+ <link href="https://fonts.googleapis.com/css2?family=Outfit:wght@300;400;500;600;700&display=swap" rel="stylesheet">
26
+
27
+ <style>
28
+ :root {
29
+ --sl-font-sans: 'Outfit', -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
30
+ --brand-primary: #10b981;
31
+ /* Clinical Green */
32
+ --brand-trust: #0f172a;
33
+ /* Deep Navy */
34
+ --bg-subtle: #f8fafc;
35
+ }
36
+
37
+ body {
38
+ font-family: var(--sl-font-sans);
39
+ background: linear-gradient(180deg, #ffffff 0%, var(--bg-subtle) 100%);
40
+ min-height: 100vh;
41
+ color: var(--brand-trust);
42
+ }
43
+
44
+ .container {
45
+ max-width: 1000px;
46
+ margin: 0 auto;
47
+ padding: 3rem 1.5rem;
48
+ }
49
+
50
+ /* Glassmorphism Effect */
51
+ .glass-overlay {
52
+ background: rgba(15, 23, 42, 0.85);
53
+ backdrop-filter: blur(8px);
54
+ -webkit-backdrop-filter: blur(8px);
55
+ }
56
+
57
+ /* Premium Card */
58
+ .card-premium {
59
+ border: 1px solid var(--sl-color-neutral-200);
60
+ box-shadow: 0 4px 6px -1px rgb(0 0 0 / 0.05), 0 2px 4px -2px rgb(0 0 0 / 0.05);
61
+ transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
62
+ background: white;
63
+ border-radius: var(--sl-border-radius-large);
64
+ }
65
+
66
+ .card-premium:hover {
67
+ transform: translateY(-2px);
68
+ box-shadow: 0 10px 15px -3px rgb(0 0 0 / 0.07);
69
+ border-color: var(--sl-color-primary-200);
70
+ }
71
+
72
+ .card-premium.main-upload-card {
73
+ box-shadow: var(--sl-shadow-large);
74
+ }
75
+
76
+ .card-premium.main-upload-card:hover {
77
+ box-shadow: var(--sl-shadow-large);
78
+ transform: none;
79
+ }
80
+
81
+ .main-upload-card::part(body) {
82
+ padding: 0;
83
+ width: 100%;
84
+ }
85
+
86
+ /* Enhanced Upload Zone */
87
+ .upload-zone {
88
+ background: linear-gradient(135deg, var(--sl-color-primary-50) 0%, white 100%);
89
+ border: 2px dashed var(--sl-color-primary-300) !important;
90
+ border-radius: var(--sl-border-radius-large);
91
+ transition: all 0.3s ease;
92
+ }
93
+
94
+ .upload-zone:hover {
95
+ border-color: var(--sl-color-primary-500) !important;
96
+ background: linear-gradient(135deg, var(--sl-color-primary-100) 0%, white 100%);
97
+ }
98
+
99
+ .timeline-container img {
100
+ transition: transform 0.3s ease;
101
+ }
102
+
103
+ .timeline-container .timeline-item:hover img {
104
+ transform: scale(1.02);
105
+ }
106
+
107
+ .analysis-card::part(base),
108
+ .timeline-item::part(base) {
109
+ box-shadow: var(--sl-shadow-large);
110
+ border: 2px solid var(--sl-color-neutral-200);
111
+ background-color: #ffffff;
112
+ /*transition: all 0.3s ease;*/
113
+ }
114
+
115
+ .interactive-result {
116
+ transition: all 0.3s ease;
117
+ }
118
+
119
+ .interactive-result:hover {
120
+ border-color: var(--sl-color-primary-300) !important;
121
+ box-shadow: var(--sl-shadow-large) !important;
122
+ }
123
+
124
+ .timeline-group::part(base) {
125
+ box-shadow: none;
126
+ background: transparent;
127
+ border: none;
128
+ }
129
+
130
+ .timeline-group::part(header) {
131
+ border-bottom: none;
132
+ padding: 0 0 1rem 0;
133
+ }
134
+
135
+ .timeline-group::part(body) {
136
+ padding: 0;
137
+ }
138
+
139
+ .timeline-container {
140
+ position: relative;
141
+ padding-left: 2.5rem;
142
+ padding-bottom: 2rem;
143
+ display: flex;
144
+ flex-direction: column;
145
+ gap: 2rem;
146
+ }
147
+
148
+ .timeline-container::before {
149
+ content: '';
150
+ position: absolute;
151
+ left: 0.5rem;
152
+ top: 0.75rem;
153
+ bottom: 2rem;
154
+ width: 2px;
155
+ background-color: var(--sl-color-primary-200);
156
+ z-index: 0;
157
+ }
158
+
159
+ .timeline-dot {
160
+ position: absolute;
161
+ left: calc(-2rem + 1px);
162
+ top: 50%;
163
+ transform: translate(-50%, -50%);
164
+ width: 14px;
165
+ height: 14px;
166
+ border-radius: 50%;
167
+ background-color: var(--sl-color-primary-600);
168
+ box-shadow: 0 0 0 5px var(--sl-color-primary-100), 0 0 12px rgba(13, 114, 255, 0.3);
169
+ z-index: 1;
170
+ }
171
+
172
+ [x-cloak] {
173
+ display: none !important;
174
+ }
175
+
176
+ /* Floating Action Button */
177
+ .fab {
178
+ position: fixed;
179
+ bottom: 2.5rem;
180
+ right: 2.5rem;
181
+ width: 60px;
182
+ height: 60px;
183
+ border-radius: 50%;
184
+ background-color: var(--sl-color-primary-600);
185
+ color: white;
186
+ display: flex;
187
+ align-items: center;
188
+ justify-content: center;
189
+ box-shadow: 0 10px 25px -5px rgba(16, 185, 129, 0.4);
190
+ cursor: pointer;
191
+ z-index: 1000;
192
+ transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
193
+ border: none;
194
+ text-decoration: none;
195
+ }
196
+
197
+ .fab:hover {
198
+ background-color: var(--sl-color-primary-700);
199
+ transform: scale(1.15) rotate(10deg);
200
+ box-shadow: 0 15px 30px -5px rgba(16, 185, 129, 0.5);
201
+ }
202
+
203
+ .fab:active {
204
+ transform: scale(0.95);
205
+ }
206
+
207
+ .fab sl-icon {
208
+ font-size: 1.5rem;
209
+ }
210
+
211
+ #header {
212
+ margin-bottom: 3rem;
213
+ text-align: center;
214
+ }
215
+
216
+ .analysis-history-container {
217
+ border-top: 1px solid var(--sl-color-neutral-100);
218
+ padding: 0 2rem 2rem 2rem;
219
+ }
220
+
221
+ .analysis-image-container {
222
+ position: relative;
223
+ width: 300px;
224
+ height: 300px;
225
+ background: var(--sl-color-neutral-100);
226
+ display: flex;
227
+ align-items: center;
228
+ justify-content: center;
229
+ border-radius: var(--sl-border-radius-medium);
230
+ overflow: hidden;
231
+ cursor: pointer;
232
+ }
233
+
234
+ .analysis-layout {
235
+ display: flex;
236
+ gap: 1.5rem;
237
+ align-items: flex-start;
238
+ flex-wrap: nowrap;
239
+ }
240
+
241
+ .analysis-details-column {
242
+ flex-grow: 1;
243
+ min-width: 0;
244
+ }
245
+
246
+ @media (max-width: 600px) {
247
+
248
+
249
+ .container {
250
+ padding: 1.5rem 0;
251
+ }
252
+
253
+ #header {
254
+ margin-bottom: 1.5rem;
255
+ }
256
+
257
+ .analysis-layout {
258
+ flex-direction: column;
259
+ align-items: stretch;
260
+ }
261
+
262
+ .analysis-image-container {
263
+ width: 100%;
264
+ height: auto;
265
+ aspect-ratio: 1;
266
+ }
267
+
268
+ .card-premium.main-upload-card,
269
+ .card-premium.main-upload-card:hover {
270
+ border: none;
271
+ box-shadow: none;
272
+ border-radius: 0;
273
+ margin-bottom: 0 !important;
274
+ }
275
+
276
+ sl-card.timeline-group {
277
+ box-shadow: none;
278
+ border: none;
279
+ }
280
+
281
+ .card-premium.main-upload-card::part(base) {
282
+ border: none;
283
+ box-shadow: none;
284
+ border-radius: 0;
285
+ }
286
+
287
+ .analysis-history-container {
288
+ border-top: none;
289
+ padding: 0;
290
+ }
291
+
292
+ .timeline-container {
293
+ border: none;
294
+ box-shadow: none;
295
+ padding-left: 0;
296
+ }
297
+
298
+ .timeline-container::before {
299
+ display: none;
300
+ }
301
+
302
+ .timeline-dot {
303
+ position: static;
304
+ transform: none;
305
+ }
306
+
307
+ .timeline-item,
308
+ .timeline-item::part(base) {
309
+ border: none;
310
+ box-shadow: none;
311
+ border-radius: 0;
312
+ }
313
+
314
+ .fab {
315
+ bottom: 1.5rem;
316
+ right: 1.5rem;
317
+ width: 54px;
318
+ height: 54px;
319
+ }
320
+ }
321
+ </style>
322
+ </head>
323
+
324
+ <body x-data="dermatologApp()">
325
+
326
+ <div class="container">
327
+
328
+ <!-- Header -->
329
+ <div id="header">
330
+ <div
331
+ style="display: flex; align-items: center; justify-content: center; gap: 0.75rem; margin-bottom: 0.5rem;">
332
+ <sl-icon name="activity" style="font-size: 2rem; color: var(--brand-primary);"></sl-icon>
333
+ <h1 style="margin: 0; font-weight: 700; letter-spacing: -0.025em; font-size: 1.75rem;">Dermatolog <span
334
+ style="color: var(--sl-color-primary-600)">AI Scan</span></h1>
335
+ </div>
336
+ <p style="color: var(--sl-color-neutral-500); max-width: 500px; margin: 0 auto; line-height: 1.6;">
337
+ A privacy-first, free, and easy-to-use skin lesion scan app powered by latest AI models.
338
+ </p>
339
+ </div>
340
+
341
+ <!-- Drag & Drop Upload Zone (Clean) -->
342
+ <sl-card class="card-premium main-upload-card" style="margin-bottom: 1.5rem; overflow: hidden; width: 100%;">
343
+ <div class="upload-zone" @dragover.prevent="dragover = true" @dragleave.prevent="dragover = false"
344
+ @drop.prevent="handleDrop($event)" @click="$refs.fileInput.click()"
345
+ style="text-align: center; padding: 4rem 2rem 2rem 2rem; cursor: pointer;">
346
+
347
+ <div style="display: flex; flex-direction: column; align-items: center; gap: 1.25rem;">
348
+ <div
349
+ style="background: white; width: 64px; height: 64px; border-radius: 50%; display: flex; align-items: center; justify-content: center; box-shadow: var(--sl-shadow-sm);">
350
+ <sl-icon name="search" style="font-size: 2rem; color: var(--sl-color-primary-600);"></sl-icon>
351
+ </div>
352
+
353
+ <div>
354
+ <h3 style="margin: 0 0 0.25rem 0; font-size: 1.25rem; font-weight: 600;">Secure Image Analysis
355
+ </h3>
356
+ <p style="color: var(--sl-color-neutral-500); font-size: 0.95rem; margin: 0;">
357
+ Drag photos here or click to browse
358
+ </p>
359
+ </div>
360
+
361
+ <div style="font-size: 0.75rem; color: var(--sl-color-neutral-400);">
362
+ <sl-icon name="keyboard" style="vertical-align: middle;"></sl-icon> Press Ctrl+V to paste images
363
+ </div>
364
+
365
+ <!-- Action Buttons Inside Zone -->
366
+ <div style="display: flex; gap: 1rem; flex-wrap: wrap; justify-content: center; margin-top: 0.5rem;"
367
+ @click.stop>
368
+ <sl-button variant="primary" pill @click.stop="$refs.cameraInput.click()"
369
+ style="min-width: 140px;">
370
+ <sl-icon slot="prefix" name="camera"></sl-icon>
371
+ Capture Image
372
+ </sl-button>
373
+ <sl-button pill @click.stop="$refs.fileInput.click()">
374
+ <sl-icon slot="prefix" name="image"></sl-icon>
375
+ Library
376
+ </sl-button>
377
+ </div>
378
+ </div>
379
+
380
+ </div>
381
+
382
+ <!-- Hidden Inputs -->
383
+ <input type="file" x-ref="fileInput" multiple accept="image/*" style="display: none;"
384
+ @change="handleFiles($event.target.files)" @click.stop>
385
+ <input type="file" x-ref="cameraInput" accept="image/*" capture="environment" style="display: none;"
386
+ @change="handleFiles($event.target.files)" @click.stop>
387
+
388
+ <!-- Analysis History Inside Card -->
389
+ <div class="analysis-history-container">
390
+ <!-- Timeline View -->
391
+ <div
392
+ style="display: flex; justify-content: space-between; align-items: center; padding-top: 1.5rem; margin-bottom: 1rem;">
393
+ <h3 x-show="timeline.length > 0" style="margin: 0; font-size: 1.1rem; font-weight: 600;">Analysis
394
+ History</h3>
395
+ <template x-if="timeline.length > 0">
396
+ <sl-button variant="danger" outline pill size="small" @click="clearSession">
397
+ <sl-icon slot="prefix" name="trash"></sl-icon>
398
+ Clear History
399
+ </sl-button>
400
+ </template>
401
+ </div>
402
+
403
+ <template x-if="timeline.length === 0">
404
+ <div style="text-align: center; padding: 4rem 0; color: var(--sl-color-neutral-400);">
405
+ <sl-icon name="image" style="font-size: 3rem; margin-bottom: 1rem; opacity: 0.3;"></sl-icon>
406
+ <div style="font-size: 1.1rem; font-weight: 500;">History Empty</div>
407
+ <p style="font-size: 0.9rem; margin-top: 0.25rem;">Upload medical images to begin AI assessment.
408
+ </p>
409
+ </div>
410
+ </template>
411
+
412
+ <div class="timeline-container">
413
+ <template x-for="item in timeline" :key="item.date">
414
+
415
+ <!-- Virtual Directory (Group) -->
416
+ <template x-if="item.type === 'directory'">
417
+ <sl-card class="timeline-group">
418
+ <div slot="header"
419
+ style="display: flex; justify-content: space-between; align-items: center;">
420
+ <div style="display: flex; align-items: center; gap: 0.75rem; position: relative;">
421
+ <div class="timeline-dot"></div>
422
+ <strong x-text="item.date"
423
+ style="font-size: 1.15rem; color: var(--sl-color-neutral-800);"></strong>
424
+ </div>
425
+ <sl-badge variant="primary" pill
426
+ x-text="item.items.length === 1 ? '1 Photo' : item.items.length + ' Photos'"></sl-badge>
427
+ </div>
428
+
429
+ <!-- View Mode: Grid (Default) -->
430
+ <template x-if="Object.keys(analysisResults).length === 0">
431
+ <div
432
+ style="display: grid; grid-template-columns: repeat(auto-fill, minmax(100px, 1fr)); gap: 0.5rem;">
433
+ <template x-for="photo in item.items" :key="photo.id">
434
+ <div style="position: relative; cursor: pointer;"
435
+ :data-analyzed="!!analysisResults[photo.id]">
436
+ <img :src="photo.local_content || '/api/photos/' + photo.id + '/content'"
437
+ style="width: 100%; aspect-ratio: 1; object-fit: cover; border-radius: var(--sl-border-radius-medium);"
438
+ @click="openEditModal(photo)">
439
+
440
+ <!-- Processing Overlay -->
441
+ <template x-if="currentAnalysisId === photo.id">
442
+ <div
443
+ style="position: absolute; inset: 0; background: rgba(255,255,255,0.7); display: flex; align-items: center; justify-content: center; border-radius: var(--sl-border-radius-medium);">
444
+ <sl-spinner
445
+ style="font-size: 2rem; --track-width: 4px; color: var(--sl-color-primary-600);"></sl-spinner>
446
+ </div>
447
+ </template>
448
+
449
+ <!-- Waiting Overlay -->
450
+ <template
451
+ x-if="loading && !analysisResults[photo.id] && currentAnalysisId !== photo.id">
452
+ <div
453
+ style="position: absolute; inset: 0; background: rgba(0,0,0,0.5); display: flex; align-items: center; justify-content: center; color: white; border-radius: var(--sl-border-radius-medium);">
454
+ <sl-icon name="hourglass-split"
455
+ style="font-size: 2rem; opacity: 0.8;"></sl-icon>
456
+ </div>
457
+ </template>
458
+
459
+ <!-- Primary Result Overlay -->
460
+ <template x-if="analysisResults[photo.id]">
461
+ <div
462
+ style="position: absolute; bottom: 0; left: 0; right: 0; background: rgba(0,0,0,0.85); color: white; padding: 2px; font-size: 0.6rem; text-align: center; border-bottom-left-radius: var(--sl-border-radius-medium); border-bottom-right-radius: var(--sl-border-radius-medium);">
463
+ <div x-text="analysisResults[photo.id].interpretation ? analysisResults[photo.id].interpretation.annotation : analysisResults[photo.id].prediction.label"
464
+ :style="'color: ' + (analysisResults[photo.id].interpretation ? getInterpretationColor(analysisResults[photo.id].interpretation.color_hint) : 'white')"
465
+ style="white-space: nowrap; overflow: hidden; text-overflow: ellipsis; font-weight: bold;">
466
+ </div>
467
+ </div>
468
+ </template>
469
+
470
+ </div>
471
+ </template>
472
+ </div>
473
+ </template>
474
+
475
+ <!-- View Mode: List (Analysis Mode) -->
476
+ <template x-if="Object.keys(analysisResults).length > 0">
477
+ <div style="display: flex; flex-direction: column; gap: 1rem;">
478
+ <template x-for="photo in item.items" :key="photo.id">
479
+ <sl-card class="analysis-card">
480
+ <div class="analysis-layout">
481
+
482
+ <!-- Image Pair Container -->
483
+ <div style="display: flex; gap: 1rem;">
484
+ <!-- Original Image -->
485
+ <div style="text-align: center;"
486
+ :data-analyzed="!!analysisResults[photo.id]">
487
+ <div class="analysis-image-container"
488
+ @click="openEditModal(photo)">
489
+ <img :src="photo.local_content || '/api/photos/' + photo.id + '/content'"
490
+ :alt="'Original Image: ' + photo.filename"
491
+ style="width: 100%; height: 100%; object-fit: contain;">
492
+
493
+ <!-- Processing Overlay -->
494
+ <template x-if="currentAnalysisId === photo.id">
495
+ <div
496
+ style="position: absolute; inset: 0; background: rgba(255,255,255,0.7); display: flex; align-items: center; justify-content: center;">
497
+ <sl-spinner
498
+ style="font-size: 3rem; --track-width: 6px; color: var(--sl-color-primary-600);"></sl-spinner>
499
+ </div>
500
+ </template>
501
+
502
+ <!-- Waiting Overlay -->
503
+ <template
504
+ x-if="loading && !analysisResults[photo.id] && currentAnalysisId !== photo.id">
505
+ <div
506
+ style="position: absolute; inset: 0; background: rgba(0,0,0,0.5); display: flex; align-items: center; justify-content: center; color: white;">
507
+ <sl-icon name="hourglass-split"
508
+ style="font-size: 3rem; opacity: 0.8;"></sl-icon>
509
+ </div>
510
+ </template>
511
+ </div>
512
+ <div style="margin-top: 0.5rem; color: var(--sl-color-neutral-500); font-size: 0.8rem;"
513
+ x-text="photo.filename"></div>
514
+ </div>
515
+ </div>
516
+
517
+ <!-- Analysis Details -->
518
+ <div class="analysis-details-column">
519
+ <template x-if="analysisResults[photo.id]">
520
+ <div
521
+ style="display: flex; flex-direction: column; gap: 1rem;">
522
+ <div
523
+ style="display: flex; flex-direction: column; gap: 1rem;">
524
+ <!-- Simplified Main Result -->
525
+ <div
526
+ :style="'padding: 1.5rem; border-radius: var(--sl-border-radius-large); background: white; transition: all 0.4s ease; box-shadow: var(--sl-shadow-large); border: 2px solid ' + (analysisResults[photo.id] && analysisResults[photo.id].interpretation ? getInterpretationColor(analysisResults[photo.id].interpretation.color_hint) : 'var(--sl-color-primary-300)')">
527
+ <div
528
+ style="display: flex; align-items: center; gap: 1rem;">
529
+ <div
530
+ :style="'width: 48px; height: 48px; border-radius: 50%; display: flex; align-items: center; justify-content: center; background: ' + (analysisResults[photo.id] && analysisResults[photo.id].interpretation ? getInterpretationColor(analysisResults[photo.id].interpretation.color_hint) + '22' : 'var(--sl-color-primary-50)')">
531
+ <sl-icon
532
+ :name="analysisResults[photo.id]?.interpretation ? getInterpretationIcon(analysisResults[photo.id].interpretation.color_hint) : 'activity'"
533
+ :style="'font-size: 1.5rem; color: ' + (analysisResults[photo.id]?.interpretation ? getInterpretationColor(analysisResults[photo.id].interpretation.color_hint) : 'var(--sl-color-primary-600)')"></sl-icon>
534
+ </div>
535
+ <div>
536
+ <div
537
+ style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 0.25rem;">
538
+ <div
539
+ style="font-size: 0.8rem; color: var(--sl-color-neutral-500); text-transform: uppercase; font-weight: 700; letter-spacing: 0.05em;">
540
+ AI Scan result
541
+ </div>
542
+ </div>
543
+ <h3 style="margin: 0; font-size: 1.25rem; font-weight: 600;"
544
+ :style="'color: ' + (analysisResults[photo.id] && analysisResults[photo.id].interpretation ? getInterpretationColor(analysisResults[photo.id].interpretation.color_hint) : 'var(--sl-color-neutral-900)')"
545
+ x-text="analysisResults[photo.id].interpretation ? analysisResults[photo.id].interpretation.annotation : analysisResults[photo.id].prediction.label">
546
+ </h3>
547
+
548
+ <template
549
+ x-if="analysisResults[photo.id] && analysisResults[photo.id].prediction && !analysisResults[photo.id].prediction.is_healthy">
550
+ <div
551
+ style="display: flex; align-items: center; gap: 0.5rem; margin-top: 0.25rem;">
552
+ <span
553
+ style="font-size: 0.9rem; font-weight: 600; color: var(--sl-color-neutral-600);">
554
+ Top Classification: <span
555
+ x-text="analysisResults[photo.id].prediction.label"
556
+ style="color: var(--sl-color-neutral-900);"></span>
557
+ </span>
558
+ <span
559
+ style="color: var(--sl-color-neutral-400); font-size: 0.8rem;"
560
+ x-text="'(' + Math.round(analysisResults[photo.id].prediction.score * 100) + '%)'"></span>
561
+
562
+ <template
563
+ x-if="analysisResults[photo.id] && analysisResults[photo.id].interpretation">
564
+ <sl-badge
565
+ :variant="getBadgeVariant(analysisResults[photo.id].interpretation.confidence_color)"
566
+ pill size="small"
567
+ style="margin-left: 0.5rem;">
568
+ <span
569
+ x-text="analysisResults[photo.id].interpretation.confidence_label"></span>
570
+ </sl-badge>
571
+ </template>
572
+ </div>
573
+ </template>
574
+ </div>
575
+ </div>
576
+
577
+ </div>
578
+
579
+ <!-- All Predictions Section (Always available if analyzed) -->
580
+ <sl-details class="interactive-result"
581
+ style="--border-width: 0; font-size: 0.8rem; margin-top: 0.5rem; border: 1px solid var(--sl-color-neutral-200); border-radius: var(--sl-border-radius-medium); overflow: hidden; background: var(--sl-color-neutral-50);">
582
+ <div slot="summary"
583
+ style="display: flex; align-items: center; gap: 0.5rem; font-weight: 600; color: var(--sl-color-neutral-700);">
584
+ <sl-icon name="list-stars"></sl-icon>
585
+ Detailed Predictions & Confidences
586
+ </div>
587
+ <div
588
+ style="padding: 0.75rem; display: flex; flex-direction: column; gap: 0.5rem; background: var(--sl-color-neutral-50);">
589
+
590
+ <div
591
+ style="font-size: 0.7rem; text-transform: uppercase; color: var(--sl-color-neutral-500); letter-spacing: 0.05em; margin-bottom: 0.25rem;">
592
+ <span
593
+ x-text="analysisResults[photo.id].primary_name || 'Primary Prediction'"></span>
594
+ </div>
595
+ <template
596
+ x-for="p in (analysisResults[photo.id].primary || [])"
597
+ :key="p.label">
598
+ <div
599
+ style="display: flex; justify-content: space-between; align-items: center; border-bottom: 1px solid var(--sl-color-neutral-200); padding: 0.5rem 0;">
600
+ <span x-text="p.label"
601
+ style="color: var(--sl-color-neutral-800); font-weight: 500;"></span>
602
+ <sl-badge
603
+ :variant="p.score > 0.5 ? 'primary' : 'neutral'"
604
+ size="small" pill
605
+ x-text="(p.score * 100).toFixed(1) + '%'"></sl-badge>
606
+ </div>
607
+ </template>
608
+ </div>
609
+ </sl-details>
610
+
611
+ <!-- Saliency Map Toggle -->
612
+ <sl-details class="interactive-result"
613
+ summary="View Saliency Map"
614
+ @sl-show="fetchSaliency(photo)"
615
+ style="--border-width: 0; --background-color: var(--sl-color-neutral-50); font-size: 0.8rem; border: 1px solid var(--sl-color-neutral-200); border-radius: var(--sl-border-radius-medium); margin-top: 0.5rem;">
616
+ <div style="padding: 0.5rem 0;">
617
+ <template
618
+ x-if="!analysisResults[photo.id].saliency_base64">
619
+ <div
620
+ style="text-align: center; padding: 1rem;">
621
+ <sl-spinner
622
+ style="font-size: 1.5rem;"></sl-spinner>
623
+ <div
624
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500);">
625
+ Computing Grad-CAM Heatmap ...
626
+ </div>
627
+ </div>
628
+ </template>
629
+ <template
630
+ x-if="analysisResults[photo.id].saliency_base64">
631
+ <div>
632
+ <div
633
+ style="position: relative; aspect-ratio: 1; background: var(--sl-color-neutral-100); border-radius: 4px; overflow: hidden;">
634
+ <img :src="analysisResults[photo.id].saliency_base64.startsWith('data:') ? analysisResults[photo.id].saliency_base64 : 'data:image/jpeg;base64,' + analysisResults[photo.id].saliency_base64"
635
+ style="width: 100%; height: 100%; object-fit: contain;"
636
+ loading="lazy">
637
+ </div>
638
+ <p
639
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500); line-height: 1.4;">
640
+ Red/orange areas indicate
641
+ regions
642
+ that
643
+ most influenced
644
+ the
645
+ primary model's classification.
646
+ </p>
647
+ </div>
648
+ </template>
649
+ </div>
650
+ </sl-details>
651
+
652
+ <div style="color: var(--sl-color-neutral-400); font-size: 0.75rem; margin-top: 0.5rem;"
653
+ x-text="'Scan executed on: ' + new Date(analysisResults[photo.id].date).toLocaleString()">
654
+ </div>
655
+ </div>
656
+ </template>
657
+
658
+ <template x-if="!analysisResults[photo.id]">
659
+ <div
660
+ style="color: var(--sl-color-neutral-400); font-style: italic;">
661
+ Pending analysis...
662
+ </div>
663
+ </template>
664
+ </div>
665
+ </div>
666
+ </sl-card>
667
+ </template>
668
+ </div>
669
+ </template>
670
+ </sl-card>
671
+ </template>
672
+
673
+ <!-- Single Photo -->
674
+ <template x-if="item.type === 'photo'">
675
+ <sl-card class="card-premium timeline-item" style="margin-bottom: 1.5rem;"
676
+ :data-analyzed="!!analysisResults[item.data.id]">
677
+ <div style="display: flex; gap: 1.5rem; padding: 0.5rem;">
678
+ <!-- Evidence Display (Thumbnail + optional Saliency) -->
679
+ <div style="display: flex; gap: 0.75rem;">
680
+ <!-- Thumbnail -->
681
+ <div style="width: 130px; aspect-ratio: 1; cursor: pointer; position: relative; overflow: hidden; border-radius: var(--sl-border-radius-medium); border: 1px solid var(--sl-color-neutral-200);"
682
+ @click="openEditModal(item.data)">
683
+ <img :src="item.data.local_content || '/api/photos/' + item.data.id + '/content'"
684
+ style="width: 100%; height: 100%; object-fit: cover;">
685
+
686
+ <!-- Result Overlay (Glassmorphism) -->
687
+ <template x-if="analysisResults[item.data.id]">
688
+ <div class="glass-overlay"
689
+ style="position: absolute; bottom: 0; left: 0; right: 0; color: white; padding: 6px; font-size: 0.7rem; text-align: center;">
690
+ <div x-text="analysisResults[item.data.id].interpretation ? analysisResults[item.data.id].interpretation.annotation : analysisResults[item.data.id].prediction.label"
691
+ :style="'color: ' + (analysisResults[item.data.id].interpretation ? getInterpretationColor(analysisResults[item.data.id].interpretation.color_hint) : 'white')"
692
+ style="font-weight: 700; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; text-transform: uppercase; letter-spacing: 0.02em;">
693
+ </div>
694
+ </div>
695
+ </template>
696
+ </div>
697
+
698
+ <!-- Lazy Loaded Saliency Heatmap (Small compare view) -->
699
+ <template
700
+ x-if="analysisResults[item.data.id] && analysisResults[item.data.id].saliency_base64">
701
+ <div
702
+ style="width: 130px; aspect-ratio: 1; border-radius: var(--sl-border-radius-medium); border: 1px solid var(--sl-color-neutral-200); background: var(--sl-color-neutral-100); overflow: hidden;">
703
+ <img :src="analysisResults[item.data.id].saliency_base64.startsWith('data:') ? analysisResults[item.data.id].saliency_base64 : 'data:image/jpeg;base64,' + analysisResults[item.data.id].saliency_base64"
704
+ style="width: 100%; height: 100%; object-fit: contain;">
705
+ </div>
706
+ </template>
707
+ </div>
708
+
709
+ <!-- Diagnostic Info -->
710
+ <div
711
+ style="flex-grow: 1; display: flex; flex-direction: column; justify-content: center;">
712
+ <div style="display: flex; justify-content: space-between; align-items: start;">
713
+ <div style="width: 100%;">
714
+ <div
715
+ style="display: flex; align-items: center; justify-content: space-between; margin-bottom: 0.75rem;">
716
+ <span
717
+ style="font-size: 0.75rem; font-weight: 600; text-transform: uppercase; letter-spacing: 0.05em; color: var(--sl-color-neutral-400);"
718
+ x-text="item.date"></span>
719
+ <sl-badge variant="neutral" pill size="small"
720
+ x-text="item.data.filename" style="opacity: 0.7;"></sl-badge>
721
+ </div>
722
+
723
+ <!-- Results List -->
724
+ <template x-if="analysisResults[item.data.id]">
725
+ <div style="display: flex; flex-direction: column; gap: 0.75rem;">
726
+ <div
727
+ :style="'display: flex; align-items: center; gap: 0.75rem; background: white; padding: 1rem; border-radius: var(--sl-border-radius-medium); box-shadow: var(--sl-shadow-medium); transition: all 0.4s ease; border: 2px solid ' + (analysisResults[item.data.id].interpretation ? getInterpretationColor(analysisResults[item.data.id].interpretation.color_hint) : 'var(--sl-color-primary-300)')">
728
+ <div :style="'background: ' + (analysisResults[item.data.id].prediction.is_healthy ? 'var(--sl-color-success-100)' : 'var(--sl-color-primary-100)')"
729
+ style="width: 40px; height: 40px; border-radius: 50%; display: flex; align-items: center; justify-content: center;">
730
+ <sl-icon
731
+ :name="analysisResults[item.data.id].prediction.is_healthy ? 'check-lg' : 'activity'"
732
+ :style="'font-size: 1.25rem; color: ' + (analysisResults[item.data.id].prediction.is_healthy ? 'var(--sl-color-success-600)' : 'var(--sl-color-primary-600)')"></sl-icon>
733
+ </div>
734
+
735
+ <div style="flex-grow: 1;">
736
+ <div
737
+ style="display: flex; align-items: center; gap: 0.5rem;">
738
+ <span style="font-weight: 700; font-size: 1.1rem;"
739
+ :style="'color: ' + (analysisResults[item.data.id].interpretation ? getInterpretationColor(analysisResults[item.data.id].interpretation.color_hint) : 'inherit')"
740
+ x-text="analysisResults[item.data.id].interpretation ? analysisResults[item.data.id].interpretation.annotation : analysisResults[item.data.id].prediction.label"></span>
741
+
742
+ <template
743
+ x-if="analysisResults[item.data.id].interpretation">
744
+ <sl-badge
745
+ :variant="analysisResults[item.data.id].interpretation.confidence_color === 'red' ? 'danger' : (analysisResults[item.data.id].interpretation.confidence_color === 'yellow' ? 'warning' : 'success')"
746
+ pill size="small"
747
+ x-text="analysisResults[item.data.id].interpretation.confidence_label"></sl-badge>
748
+ </template>
749
+ </div>
750
+ <div
751
+ style="font-size: 0.85rem; color: var(--sl-color-neutral-500); margin-top: 2px;">
752
+ <span
753
+ x-text="analysisResults[item.data.id].prediction.is_healthy ? 'No immediate concerns detected' : 'Inconclusive or requires review'"></span>
754
+ <span
755
+ x-text="' • Conf: ' + Math.round(analysisResults[item.data.id].prediction.score * 100) + '%'"></span>
756
+ </div>
757
+ </div>
758
+ </div>
759
+ <!-- All Predictions Section (Always available if analyzed) -->
760
+ <sl-details class="interactive-result"
761
+ style="--border-width: 0; font-size: 0.8rem; margin-top: 0.5rem; border: 1px solid var(--sl-color-neutral-200); border-radius: var(--sl-border-radius-medium); overflow: hidden; background: var(--sl-color-neutral-50);">
762
+ <div slot="summary"
763
+ style="display: flex; align-items: center; gap: 0.5rem; font-weight: 600; color: var(--sl-color-neutral-700);">
764
+ <sl-icon name="list-stars"></sl-icon>
765
+ Detailed Predictions & Confidences
766
+ </div>
767
+ <div
768
+ style="padding: 0.75rem; display: flex; flex-direction: column; gap: 0.5rem; background: var(--sl-color-neutral-50);">
769
+
770
+ <div
771
+ style="font-size: 0.7rem; text-transform: uppercase; color: var(--sl-color-neutral-500); letter-spacing: 0.05em; margin-bottom: 0.25rem;">
772
+ <span
773
+ x-text="analysisResults[item.data.id].primary_name || 'Primary Prediction'"></span>
774
+ </div>
775
+ <template
776
+ x-for="p in (analysisResults[item.data.id].primary || [])"
777
+ :key="p.label">
778
+ <div
779
+ style="display: flex; justify-content: space-between; align-items: center; border-bottom: 1px solid var(--sl-color-neutral-200); padding: 0.5rem 0;">
780
+ <span x-text="p.label"
781
+ style="color: var(--sl-color-neutral-800); font-weight: 500;"></span>
782
+ <sl-badge
783
+ :variant="p.score > 0.5 ? 'primary' : 'neutral'"
784
+ size="small" pill
785
+ x-text="(p.score * 100).toFixed(1) + '%'"></sl-badge>
786
+ </div>
787
+ </template>
788
+ </div>
789
+ </sl-details>
790
+
791
+ <!-- Saliency Map Toggle -->
792
+ <sl-details class="interactive-result"
793
+ summary="View Grad-CAM Saliency Map"
794
+ @sl-show="fetchSaliency(item.data)"
795
+ style="--border-width: 0; --background-color: var(--sl-color-neutral-50); font-size: 0.8rem; border: 1px solid var(--sl-color-neutral-200); border-radius: var(--sl-border-radius-medium); margin-top: 0.5rem;">
796
+ <div style="padding: 0.5rem 0;">
797
+ <template
798
+ x-if="!analysisResults[item.data.id].saliency_base64">
799
+ <div style="text-align: center; padding: 1rem;">
800
+ <sl-spinner
801
+ style="font-size: 1.5rem;"></sl-spinner>
802
+ <div
803
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500);">
804
+ Computing Grad-CAM Heatmap ...
805
+ </div>
806
+ </div>
807
+ </template>
808
+ <template
809
+ x-if="analysisResults[item.data.id].saliency_base64">
810
+ <div>
811
+ <div
812
+ style="position: relative; aspect-ratio: 1; background: var(--sl-color-neutral-100); border-radius: 4px; overflow: hidden;">
813
+ <img :src="analysisResults[item.data.id].saliency_base64.startsWith('data:') ? analysisResults[item.data.id].saliency_base64 : 'data:image/jpeg;base64,' + analysisResults[item.data.id].saliency_base64"
814
+ style="width: 100%; height: 100%; object-fit: contain;"
815
+ loading="lazy">
816
+ </div>
817
+ <p
818
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500); line-height: 1.4;">
819
+ Red/orange areas indicate regions
820
+ that
821
+ most influenced
822
+ the
823
+ primary model's classification.
824
+ </p>
825
+ </div>
826
+ </template>
827
+ </div>
828
+ </sl-details>
829
+
830
+ <div
831
+ style="margin-top: 0.5rem; display: flex; justify-content: flex-end;">
832
+ <sl-button size="small" pill outline
833
+ @click="copyReport(item.data.id)">
834
+ <sl-icon slot="prefix" name="clipboard"></sl-icon>
835
+ Copy Summary
836
+ </sl-button>
837
+ </div>
838
+ </div>
839
+ </template>
840
+
841
+ <template x-if="!analysisResults[item.data.id]">
842
+ <div
843
+ style="display: flex; align-items: center; gap: 0.5rem; color: var(--sl-color-neutral-400); font-style: italic; padding: 1rem 0;">
844
+ <sl-spinner style="font-size: 1rem;"></sl-spinner>
845
+ <span>Running clinical analysis...</span>
846
+ </div>
847
+ </template>
848
+ </div>
849
+ </div>
850
+ </div>
851
+ </div>
852
+ </sl-card>
853
+ </template>
854
+
855
+ </template>
856
+ </div>
857
+ </div>
858
+ </sl-card>
859
+
860
+ <!-- Local Privacy Information -->
861
+ <div
862
+ style="display: flex; align-items: center; justify-content: center; gap: 0.5rem; margin-bottom: 2.5rem; color: var(--sl-color-neutral-500); font-size: 0.85rem; padding: 0.5rem; background: var(--sl-color-neutral-50); border-radius: var(--sl-border-radius-medium);">
863
+ <sl-icon name="shield-check" style="color: var(--sl-color-success-600); font-size: 1.1rem;"></sl-icon>
864
+ <span>Privacy Mode: Images are stored locally on your device</span>
865
+ </div>
866
+
867
+
868
+
869
+
870
+
871
+ <!-- Error -->
872
+ <template x-if="error">
873
+ <sl-alert variant="danger" open style="margin-top: 1rem;">
874
+ <sl-icon slot="icon" name="exclamation-octagon"></sl-icon>
875
+ <strong x-text="error"></strong>
876
+ </sl-alert>
877
+ </template>
878
+
879
+ <!-- Edit Modal -->
880
+ <sl-dialog label="Edit Photo Date" class="edit-dialog">
881
+ <template x-if="editingPhoto">
882
+ <div>
883
+ <img :src="editingPhoto.local_content || '/api/photos/' + editingPhoto.id + '/content'"
884
+ style="width: 100%; max-height: 200px; object-fit: contain; margin-bottom: 1rem;">
885
+ <sl-input type="date" label="Example Date" x-model="editingDate"></sl-input>
886
+ </div>
887
+ </template>
888
+ <div slot="footer" style="display: flex; justify-content: space-between;">
889
+ <sl-button variant="danger" outline @click="deletePhotoFromModal">
890
+ <sl-icon slot="prefix" name="trash"></sl-icon> Delete
891
+ </sl-button>
892
+ <sl-button variant="primary" @click="saveDate">Save</sl-button>
893
+ </div>
894
+ </sl-dialog>
895
+
896
+ <!-- Floating Action Button -->
897
+ <button class="fab" @click="$refs.fileInput.click()" title="Upload Image">
898
+ <sl-icon name="plus-lg"></sl-icon>
899
+ </button>
900
+
901
+ <!-- Debug Toggle & Developer Data -->
902
+ <div
903
+ style="margin-top: 4rem; padding: 2rem 0; border-top: 1px solid var(--sl-color-neutral-200); margin-bottom: 2rem; text-align: center;">
904
+ <sl-switch :checked="debugMode" @sl-change="toggleDebug()">
905
+ <span style="font-size: 0.8rem; color: var(--sl-color-neutral-500);">Debug Mode</span>
906
+ </sl-switch>
907
+ </div>
908
+
909
+ <!-- Debug View: All Photos -->
910
+ <template x-if="debugMode">
911
+ <div style="margin-top: 1rem; padding-top: 1rem; border-top: 1px solid var(--sl-color-neutral-200);">
912
+ <h3
913
+ style="color: var(--sl-color-neutral-500); font-size: 0.9rem; text-transform: uppercase; text-align: center; margin-bottom: 2rem;">
914
+ Debug: All Images <span
915
+ style="font-weight: normal; margin-left: 1rem; color: var(--sl-color-neutral-400);"
916
+ x-text="'Session: ' + sessionId"></span>
917
+ <span style="font-weight: normal; margin-left: 1rem; color: var(--sl-color-primary-500);"
918
+ x-text="'Model: ' + modelName"></span>
919
+ <span style="font-weight: normal; margin-left: 1rem;"
920
+ :style="yoloAvailable ? 'color: var(--sl-color-success-600);' : 'color: var(--sl-color-danger-600);'"
921
+ x-text="'YOLO: ' + (yoloAvailable ? 'Available' : 'Missing (Center Crop Fallback)')"></span>
922
+ </h3>
923
+
924
+ <!-- Debug Settings Section -->
925
+ <div
926
+ style="margin-bottom: 1.5rem; display: flex; gap: 2rem; background: var(--sl-color-neutral-50); padding: 1rem; border-radius: var(--sl-border-radius-medium); border: 1px solid var(--sl-color-neutral-200);">
927
+ <div style="text-align: left; max-width: 300px; flex: 1;">
928
+ <div
929
+ style="display: flex; justify-content: space-between; font-size: 0.75rem; color: var(--sl-color-neutral-600); margin-bottom: 0.5rem;">
930
+ <strong>Interpretation Margin</strong>
931
+ <span style="font-weight: 600; color: var(--sl-color-warning-600)"
932
+ x-text="(marginThreshold * 100).toFixed(0) + '%'"></span>
933
+ </div>
934
+ <sl-range min="0.01" max="0.20" step="0.01" :value="marginThreshold"
935
+ @sl-input="marginThreshold = $event.target.value"></sl-range>
936
+ <div style="font-size: 0.65rem; color: var(--sl-color-neutral-400); margin-top: 0.25rem;">
937
+ Threshold for "Not Clear" mixed tumor/benign results. Move this to calibrate the sensitivity
938
+ of
939
+ warnings.
940
+ </div>
941
+ </div>
942
+ <div
943
+ style="flex: 2; font-size: 0.75rem; color: var(--sl-color-neutral-500); display: flex; align-items: center;">
944
+ <sl-icon name="info-circle" style="margin-right: 0.5rem;"></sl-icon>
945
+ <span>These settings are for development and calibration. They affect how the model results are
946
+ interpreted into human-readable alerts.</span>
947
+ </div>
948
+ </div>
949
+
950
+ <div style="display: flex; gap: 1.5rem; overflow-x: auto; padding-bottom: 1rem; margin-bottom: 2rem;">
951
+ <template x-for="photo in getAllPhotos()" :key="photo.id">
952
+ <div
953
+ style="flex: 0 0 auto; text-align: center; display: flex; flex-direction: column; align-items: center; gap: 0.5rem;">
954
+ <div style="display: flex; gap: 0.5rem;">
955
+ <!-- Raw Image -->
956
+ <div style="text-align: center;">
957
+ <img :src="photo.local_content || '/api/photos/' + photo.id + '/content'"
958
+ style="width: 100px; height: 100px; object-fit: cover; border-radius: 4px; border: 1px solid var(--sl-color-neutral-300);">
959
+ <div
960
+ style="font-size: 0.6rem; color: var(--sl-color-neutral-400); margin-top: 2px;">
961
+ Raw
962
+ </div>
963
+ </div>
964
+
965
+ <!-- Processed Image -->
966
+ <template
967
+ x-if="analysisResults[photo.id] && analysisResults[photo.id].prepared_image_base64">
968
+ <div style="text-align: center;">
969
+ <img :src="analysisResults[photo.id].prepared_image_base64.startsWith('data:') ? analysisResults[photo.id].prepared_image_base64 : 'data:image/jpeg;base64,' + analysisResults[photo.id].prepared_image_base64"
970
+ style="width: 100px; height: 100px; object-fit: contain; background: var(--sl-color-neutral-100); border-radius: 4px; border: 2px solid var(--sl-color-primary-300);">
971
+ <div
972
+ style="font-size: 0.6rem; color: var(--sl-color-primary-500); margin-top: 2px; font-weight: bold;">
973
+ Target (448x448)</div>
974
+ </div>
975
+ </template>
976
+ </div>
977
+
978
+ <div style="font-size: 0.7rem; color: var(--sl-color-neutral-600); font-weight: 600;"
979
+ x-text="photo.filename"></div>
980
+ <div style="font-size: 0.6rem;"
981
+ :style="analysisResults[photo.id] ? 'color: var(--sl-color-success-600);' : 'color: var(--sl-color-warning-600);'">
982
+ <span
983
+ x-text="analysisResults[photo.id] ? 'Analysis Complete' : 'Pending Analysis'"></span>
984
+ </div>
985
+
986
+ <!-- Debug: Interpretation Logic -->
987
+ <template x-if="analysisResults[photo.id] && analysisResults[photo.id].interpretation">
988
+ <div
989
+ style="text-align: left; margin-top: 0.5rem; padding: 0.4rem; background: var(--sl-color-neutral-50); border-radius: 4px; border-left: 2px solid var(--sl-color-neutral-300); width: 100%;">
990
+ <div
991
+ style="font-size: 0.55rem; text-transform: uppercase; color: var(--sl-color-neutral-400); font-weight: bold; margin-bottom: 2px;">
992
+ Interp. Logic</div>
993
+ <template
994
+ x-for="step in analysisResults[photo.id].interpretation.computation_process">
995
+ <div
996
+ style="font-size: 0.55rem; color: var(--sl-color-neutral-500); font-family: var(--sl-font-mono); line-height: 1.2;">
997
+ » <span x-text="step"></span>
998
+ </div>
999
+ </template>
1000
+ </div>
1001
+ </template>
1002
+ </div>
1003
+ </template>
1004
+ </div>
1005
+
1006
+ <!-- Execution Times Table -->
1007
+ <template x-if="Object.values(analysisResults).some(r => r.execution_times)">
1008
+ <div style="margin-top: 1rem; margin-bottom: 2rem;">
1009
+ <sl-card>
1010
+ <div slot="header" style="font-size: 0.85rem; font-weight: 600;">System Performance (ms)
1011
+ </div>
1012
+ <table style="width: 100%; border-collapse: collapse; font-size: 0.8rem;">
1013
+ <thead>
1014
+ <tr style="text-align: left; border-bottom: 1px solid var(--sl-color-neutral-200);">
1015
+ <th style="padding: 0.5rem;">Photo</th>
1016
+ <th style="padding: 0.5rem;">Preprocess</th>
1017
+ <th style="padding: 0.5rem;">Primary AI</th>
1018
+ </tr>
1019
+ </thead>
1020
+ <tbody>
1021
+ <template x-for="photo in getAllPhotos()" :key="'time-' + photo.id">
1022
+ <template
1023
+ x-if="analysisResults[photo.id] && analysisResults[photo.id].execution_times">
1024
+ <tr style="border-bottom: 1px solid var(--sl-color-neutral-100);">
1025
+ <td style="padding: 0.5rem; color: var(--sl-color-neutral-500);"
1026
+ x-text="photo.filename"></td>
1027
+ <td style="padding: 0.5rem;"
1028
+ x-text="analysisResults[photo.id].execution_times.image_preprocess || '-'">
1029
+ </td>
1030
+ <td style="padding: 0.5rem;"
1031
+ x-text="analysisResults[photo.id].execution_times.primary_medsiglip || '-'">
1032
+ </td>
1033
+ </tr>
1034
+ </template>
1035
+ </template>
1036
+ </tbody>
1037
+ </table>
1038
+ </sl-card>
1039
+ </div>
1040
+ </template>
1041
+
1042
+ <!-- Model Log Response -->
1043
+ <template x-if="response">
1044
+ <div style="margin-bottom: 2rem;">
1045
+ <sl-card class="card-result">
1046
+ <div slot="header">
1047
+ <strong>Model Log Response</strong>
1048
+ <sl-badge pill variant="neutral" x-text="latency + ' ms'"></sl-badge>
1049
+ </div>
1050
+ <div style="white-space: pre-wrap; font-family: var(--sl-font-mono); font-size: 0.85rem;"
1051
+ x-text="response"></div>
1052
+ </sl-card>
1053
+ </div>
1054
+ </template>
1055
+
1056
+ <!-- Technical Analysis (Debug Info) Section -->
1057
+ <div style="border-top: 1px solid var(--sl-color-neutral-200); padding-top: 2rem; text-align: left;">
1058
+ <sl-details style="--border-width: 0; --background-color: transparent;">
1059
+ <span slot="summary"
1060
+ style="color: var(--sl-color-neutral-500); font-size: 0.9rem; text-transform: uppercase; font-weight: 600;">
1061
+ Detailed Inference Reports (Technical Analysis)
1062
+ </span>
1063
+
1064
+ <div
1065
+ style="display: grid; grid-template-columns: repeat(auto-fill, minmax(350px, 1fr)); gap: 1rem; margin-top: 1.5rem;">
1066
+ <template x-for="photo in getAllPhotos()" :key="'debug-' + photo.id">
1067
+ <template x-if="analysisResults[photo.id]">
1068
+ <sl-card>
1069
+ <div slot="header"
1070
+ style="display: flex; justify-content: space-between; align-items: center;">
1071
+ <strong x-text="photo.filename" style="font-size: 0.85rem;"></strong>
1072
+ <sl-badge variant="neutral" pill
1073
+ x-text="photo.id.substring(0,8)"></sl-badge>
1074
+ </div>
1075
+
1076
+ <div style="display: flex; flex-direction: column; gap: 1rem;">
1077
+ <!-- Saliency Map Toggle -->
1078
+ <sl-details summary="View Grad-CAM Saliency Map"
1079
+ @sl-show="fetchSaliency(photo)"
1080
+ style="--border-width: 0; --background-color: var(--sl-color-neutral-50); font-size: 0.8rem;">
1081
+ <div style="padding: 0.5rem 0;">
1082
+ <template x-if="!analysisResults[photo.id].saliency_base64">
1083
+ <div style="text-align: center; padding: 1rem;">
1084
+ <sl-spinner style="font-size: 1.5rem;"></sl-spinner>
1085
+ <div
1086
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500);">
1087
+ Computing Grad-CAM Heatmap (Lazy)...
1088
+ </div>
1089
+ </div>
1090
+ </template>
1091
+ <template x-if="analysisResults[photo.id].saliency_base64">
1092
+ <div>
1093
+ <div
1094
+ style="position: relative; aspect-ratio: 1; background: var(--sl-color-neutral-100); border-radius: 4px; overflow: hidden;">
1095
+ <img :src="analysisResults[photo.id].saliency_base64.startsWith('data:') ? analysisResults[photo.id].saliency_base64 : 'data:image/jpeg;base64,' + analysisResults[photo.id].saliency_base64"
1096
+ style="width: 100%; height: 100%; object-fit: contain;"
1097
+ loading="lazy">
1098
+ </div>
1099
+ <p
1100
+ style="margin-top: 0.5rem; font-size: 0.75rem; color: var(--sl-color-neutral-500); line-height: 1.4;">
1101
+ Red/orange areas indicate regions that most
1102
+ influenced the primary model's classification.
1103
+ </p>
1104
+ </div>
1105
+ </template>
1106
+ </div>
1107
+ </sl-details>
1108
+
1109
+ <div
1110
+ style="font-family: var(--sl-font-mono); font-size: 0.8rem; display: flex; flex-direction: column; gap: 0.75rem;">
1111
+ <!-- Primary Scores -->
1112
+ <div
1113
+ style="padding: 0.75rem; border-radius: 4px; border: 1px solid var(--sl-color-primary-100); background: var(--sl-color-primary-50);">
1114
+ <div style="display: flex; flex-direction: column; gap: 0.25rem;">
1115
+ <template x-for="p in (analysisResults[photo.id].primary || [])"
1116
+ :key="p.label">
1117
+ <div
1118
+ style="display: flex; justify-content: space-between; align-items: center;">
1119
+ <span x-text="p.label"
1120
+ style="color: var(--sl-color-neutral-700);"></span>
1121
+ <sl-badge
1122
+ :variant="p.score > 0.5 ? 'primary' : 'neutral'"
1123
+ size="small" pill
1124
+ x-text="(p.score * 100).toFixed(1) + '%'"></sl-badge>
1125
+ </div>
1126
+ </template>
1127
+ </div>
1128
+ </div>
1129
+
1130
+ <div style="color: var(--sl-color-neutral-500); font-size: 0.7rem;">
1131
+ Execution: <span
1132
+ x-text="analysisResults[photo.id] ? new Date(analysisResults[photo.id].date).toLocaleTimeString() : ''"></span><br>
1133
+ Strategy: <span
1134
+ x-text="analysisResults[photo.id] && analysisResults[photo.id].preprocess_strategy ? analysisResults[photo.id].preprocess_strategy.strategy : ''"></span><br>
1135
+ </div>
1136
+ </div>
1137
+ </div>
1138
+ </sl-card>
1139
+ </template>
1140
+ </template>
1141
+ </div>
1142
+ </sl-details>
1143
+ </div>
1144
+ </div>
1145
+ </template>
1146
+ </div>
1147
+ </body>
1148
+
1149
+ </html>
bin/app_restart.sh ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ echo "Stopping any existing uvicorn processes..."
3
+ pkill -f uvicorn || true
4
+
5
+ echo "Starting application server..."
6
+ # Using nohup and python -m uvicorn to ensure correct python path and persistence
7
+ nohup /Users/mstepien/Documents/dev2/py/fasts/venv/bin/python3.10 -m uvicorn app.main:app --host 0.0.0.0 --port 8000 > server.log 2>&1 &
8
+
9
+ echo "Waiting for server to be ready..."
10
+ # Simple loop to check if port 8000 is open (using curl or netcat logic via python)
11
+ for i in {1..30}; do
12
+ if curl -s http://localhost:8000/api/health >/dev/null; then
13
+ echo "Server is UP!"
14
+ exit 0
15
+ fi
16
+ echo "Waiting for server... ($i/30)"
17
+ sleep 1
18
+ done
19
+
20
+ echo "Server failed to start. Check server.log:"
21
+ tail -n 20 server.log
22
+ exit 1
bin/app_stop.sh ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ #kill <PID> && source venv/bin/activate && nohup uvicorn main:app --host 127.0.0.1 --port 8080 > server.log 2>&1 &
4
+
5
+ # Find the PID of the running uvicorn process
6
+ PID=$(ps aux | grep "uvicorn app.main:app" | grep -v grep | awk '{print $2}')
7
+
8
+ if [ -n "$PID" ]; then
9
+ echo "Stopping existing application(s) (PIDs: $PID)..."
10
+ echo "$PID" | xargs kill
11
+ sleep 2 # Wait for it to shut down
12
+ else
13
+ echo "No running application found."
14
+ fi
bin/check_models.sh ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ # Get the directory where the script is located
4
+ BIN_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
5
+ PROJECT_ROOT="$(dirname "$BIN_DIR")"
6
+
7
+ echo "Checking and downloading models for Dermatolog AI..."
8
+
9
+ # Use the same python executable as the app (or just python3)
10
+ # Based on previous turns, /usr/local/opt/python@3.8/bin/python3.8 was used
11
+ PYTHON_EXEC="/usr/local/opt/python@3.8/bin/python3.8"
12
+
13
+ if [ ! -x "$PYTHON_EXEC" ]; then
14
+ PYTHON_EXEC="python3"
15
+ fi
16
+
17
+ $PYTHON_EXEC "$BIN_DIR/download_models.py"
bin/cleanup_chromium.sh ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ # Script to clear any hanging Chromium/Chrome/Playwright processes
4
+
5
+ echo "🧹 Cleaning up hanging Chromium and test processes..."
6
+
7
+ # List of process patterns to target (case-insensitive)
8
+ TARGETS=(
9
+ #"chromium" "chrome"
10
+ "playwright" "ms-playwright")
11
+
12
+ for target in "${TARGETS[@]}"; do
13
+ # Check if any processes exist for this target (full command line match)
14
+ if pgrep -if "$target" > /dev/null; then
15
+ echo "Killing processes matching: $target"
16
+ pkill -9 -if "$target"
17
+ fi
18
+ done
19
+
20
+ # Also handle specific Playwright driver if it's hanging
21
+ if pgrep -f "playwright-core" > /dev/null; then
22
+ echo "Killing playwright-core processes..."
23
+ pkill -9 -f "playwright-core"
24
+ fi
25
+
26
+ echo "✅ Cleanup complete."
bin/deploy.sh ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ set -e
3
+
4
+
5
+ # Load .env file if it exists
6
+ if [ -f .env ]; then
7
+ export $(grep -v '^#' .env | xargs)
8
+ fi
9
+
10
+ # Check for PROJECT_ID
11
+ if [ -z "$PROJECT_ID" ] || [ "$PROJECT_ID" == "your-project-id" ]; then
12
+ echo "Error: PROJECT_ID is not set. Please set it in .env or export it."
13
+ echo "Example: export PROJECT_ID=my-gcp-project-id"
14
+ exit 1
15
+ fi
16
+
17
+ GOOGLE_CLOUD_PROJECT=$PROJECT_ID
18
+ SERVICE_NAME="dermatolog-ai-scan"
19
+ REGION="us-central1"
20
+ # We need enough memory for the model (MedSigLIP) to load.
21
+ # 4GB is the absolute minimum, 8GB is safer.
22
+ MEMORY="8Gi"
23
+ CPU="2"
24
+
25
+ echo "========================================================"
26
+ echo " Deploying $SERVICE_NAME to Cloud Run ($REGION)"
27
+ echo " Mode: Self-Contained (Local Inference)"
28
+ echo "========================================================"
29
+
30
+ # 1. Build and Submit Container (Using Cloud Build to inject build args)
31
+ echo "[1/3] Building container image..."
32
+ gcloud builds submit --config cloudbuild.yaml --substitutions=_HF_TOKEN="$HF_TOKEN",_SERVICE_NAME="$SERVICE_NAME" .
33
+
34
+ # 2. Deploy to Cloud Run
35
+ echo "[2/3] Deploying to Cloud Run..."
36
+ gcloud run deploy $SERVICE_NAME \
37
+ --image gcr.io/$GOOGLE_CLOUD_PROJECT/$SERVICE_NAME \
38
+ --region $REGION \
39
+ --platform managed \
40
+ --allow-unauthenticated \
41
+ --memory $MEMORY \
42
+ --cpu $CPU \
43
+ --timeout 300 \
44
+ --concurrency 10 \
45
+ --set-env-vars="HF_TOKEN=$HF_TOKEN"
46
+ # Note: If HF_TOKEN is not set in your local shell, this will be empty.
47
+ # The app handles missing token by falling back to public model.
48
+
49
+ echo "========================================================"
50
+ echo " Deployment Complete!"
51
+ echo "========================================================"
bin/docker-test.sh ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ # Helper script to run tests inside the Docker container
3
+
4
+ echo "Running tests in the 'app' container..."
5
+ docker compose exec app pytest "$@"
bin/download_models.py ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import sys
3
+ from huggingface_hub import snapshot_download
4
+ from dotenv import load_dotenv
5
+
6
+ # Load environment variables
7
+ load_dotenv()
8
+
9
+ MODELS = [
10
+ "google/medsiglip-448"
11
+ ]
12
+
13
+ YOLO_MODELS = [
14
+ "yolov8n.pt"
15
+ ]
16
+
17
+ def check_and_download():
18
+ token = os.environ.get("HF_TOKEN")
19
+ if not token:
20
+ print("Warning: HF_TOKEN not found in environment. Gated models like MedSigLIP may fail to download.")
21
+
22
+ success = True
23
+ for model_id in MODELS:
24
+ print(f"\n--- Checking {model_id} ---")
25
+ try:
26
+ # snackshot_download checks if files are already present and only downloads missing pieces
27
+ path = snapshot_download(
28
+ repo_id=model_id,
29
+ token=token,
30
+ local_files_only=False # Set to True if we only wanted to check, but user wants to download too
31
+ )
32
+ print(f"Model {model_id} is ready at: {path}")
33
+ except Exception as e:
34
+ print(f"Error handling {model_id}: {e}")
35
+ success = False
36
+
37
+ # Download YOLO models
38
+ try:
39
+ from ultralytics import YOLO
40
+ for yolo_model in YOLO_MODELS:
41
+ print(f"\n--- Checking YOLO {yolo_model} ---")
42
+ try:
43
+ YOLO(yolo_model)
44
+ print(f"YOLO Model {yolo_model} is ready.")
45
+ except Exception as e:
46
+ print(f"Error handling YOLO {yolo_model}: {e}")
47
+ success = False
48
+ except ImportError:
49
+ print("\nWarning: ultralytics not installed. Skipping YOLO model download.")
50
+
51
+ if success:
52
+ print("\nAll models are downloaded and verified.")
53
+ else:
54
+ print("\nSome models failed to download. Please check your HF_TOKEN and internet connection.")
55
+ sys.exit(1)
56
+
57
+ if __name__ == "__main__":
58
+ check_and_download()
bin/generate-api.sh ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ # Exit on error
4
+ set -e
5
+
6
+ echo "Generating Python models..."
7
+
8
+ # Find datamodel-codegen in path or venv
9
+ if command -v datamodel-codegen >/dev/null 2>&1; then
10
+ CODEGEN_BIN="datamodel-codegen"
11
+ elif [ -f "./venv/bin/datamodel-codegen" ]; then
12
+ CODEGEN_BIN="./venv/bin/datamodel-codegen"
13
+ else
14
+ echo "datamodel-codegen not found. Attempting to install..."
15
+ pip install datamodel-code-generator || ./venv/bin/pip install datamodel-code-generator
16
+ CODEGEN_BIN="datamodel-codegen"
17
+ if ! command -v "$CODEGEN_BIN" >/dev/null 2>&1 && [ -f "./venv/bin/datamodel-codegen" ]; then
18
+ CODEGEN_BIN="./venv/bin/datamodel-codegen"
19
+ fi
20
+ fi
21
+
22
+ if ! "$CODEGEN_BIN" --input openapi.yaml --output app/models.py; then
23
+ echo "Error: Python model generation failed. Check openapi.yaml for syntax errors."
24
+ exit 1
25
+ fi
26
+
27
+ echo "API Generation for Python Complete!"
cloudbuild.yaml ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ steps:
2
+ - name: 'gcr.io/cloud-builders/docker'
3
+ args:
4
+ - 'build'
5
+ - '--build-arg'
6
+ - 'HF_TOKEN=$_HF_TOKEN'
7
+ - '-t'
8
+ - 'europe-west1-docker.pkg.dev/$PROJECT_ID/dermatolog-scan/medgemma-app'
9
+ - '.'
10
+ images:
11
+ - 'europe-west1-docker.pkg.dev/$PROJECT_ID/dermatolog-scan/medgemma-app'
docker-compose.yml ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ version: '3.8'
2
+
3
+ services:
4
+ dermatolog-ai-scan:
5
+ build:
6
+ context: .
7
+ dockerfile: Dockerfile
8
+ command: uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
9
+ volumes:
10
+ - .:/app
11
+ ports:
12
+ - "8000:8000"
13
+ environment:
14
+
15
+ - PROJECT_ID=${PROJECT_ID}
16
+ - LOCATION=${LOCATION}
docker-test.sh ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ # Helper script to run tests inside the Docker container
3
+
4
+ echo "Running tests in the 'app' container..."
5
+ docker compose exec app pytest "$@"
generate-api.sh ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ # Exit on error
4
+ set -e
5
+
6
+ echo "Generating Python models..."
7
+
8
+ # Find datamodel-codegen in path or venv
9
+ if command -v datamodel-codegen >/dev/null 2>&1; then
10
+ CODEGEN_BIN="datamodel-codegen"
11
+ elif [ -f "./venv/bin/datamodel-codegen" ]; then
12
+ CODEGEN_BIN="./venv/bin/datamodel-codegen"
13
+ else
14
+ echo "datamodel-codegen not found. Attempting to install..."
15
+ pip install datamodel-code-generator || ./venv/bin/pip install datamodel-code-generator
16
+ CODEGEN_BIN="datamodel-codegen"
17
+ if ! command -v "$CODEGEN_BIN" >/dev/null 2>&1 && [ -f "./venv/bin/datamodel-codegen" ]; then
18
+ CODEGEN_BIN="./venv/bin/datamodel-codegen"
19
+ fi
20
+ fi
21
+
22
+ if ! "$CODEGEN_BIN" --input openapi.yaml --output app/models.py; then
23
+ echo "Error: Python model generation failed. Check openapi.yaml for syntax errors."
24
+ exit 1
25
+ fi
26
+
27
+ echo "API Generation for Python Complete!"
openapi.yaml ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ openapi: 3.0.3
2
+ info:
3
+ title: Dermatolog AI Scan
4
+ description: AI application for dermatology analysis
5
+ version: 1.0.0
6
+ paths:
7
+ /api/health:
8
+ get:
9
+ summary: Health check endpoint
10
+ operationId: health_check
11
+ responses:
12
+ '200':
13
+ description: Successful Response
14
+ content:
15
+ application/json:
16
+ schema:
17
+ $ref: '#/components/schemas/HealthCheckResponse'
18
+ components:
19
+ schemas:
20
+ HealthCheckResponse:
21
+ properties:
22
+ status:
23
+ type: string
24
+ title: Status
25
+ database:
26
+ type: string
27
+ title: Database
28
+ gcp_project:
29
+ type: string
30
+ title: Gcp Project
31
+ type: object
32
+ required:
33
+ - status
34
+ - database
35
+ title: HealthCheckResponse
package-lock.json ADDED
The diff for this file is too large to render. See raw diff
 
package.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "dermatolog-ai-scan-frontend",
3
+ "version": "1.0.0",
4
+ "engines": {
5
+ "node": ">=16.0.0"
6
+ },
7
+ "description": "Frontend JavaScript modules for Dermatolog AI Scan",
8
+ "type": "module",
9
+ "scripts": {
10
+ "test": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
11
+ "test:watch": "node --experimental-vm-modules node_modules/jest/bin/jest.js --watch",
12
+ "test:coverage": "node --experimental-vm-modules node_modules/jest/bin/jest.js --coverage",
13
+ "generate-api": "bash bin/generate-api.sh"
14
+ },
15
+ "devDependencies": {
16
+ "@jest/globals": "^29.7.0",
17
+ "jest": "^29.7.0",
18
+ "jest-environment-jsdom": "^29.7.0"
19
+ },
20
+ "jest": {
21
+ "testEnvironment": "jsdom",
22
+ "transform": {},
23
+ "testMatch": [
24
+ "**/tests/javascript/**/*.test.js"
25
+ ],
26
+ "collectCoverageFrom": [
27
+ "app/static/js/modules/**/*.js"
28
+ ],
29
+ "coverageThreshold": {
30
+ "global": {
31
+ "branches": 80,
32
+ "functions": 80,
33
+ "lines": 80,
34
+ "statements": 80
35
+ }
36
+ }
37
+ }
38
+ }
pytest.ini ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [pytest]
2
+ # Playwright configuration
3
+ # Browser tests run in headless mode by default
4
+ # Use --headed flag to run with visible browser when pytest-playwright is installed
5
+
6
+ # Asyncio configuration
7
+ # Using strict mode to avoid loop interference with non-async tests (like Playwright)
8
+ asyncio_mode = strict
9
+ asyncio_default_fixture_loop_scope = function
10
+ asyncio_default_test_loop_scope = function
11
+
12
+ # Disable anyio to avoid duplicate runner conflicts
13
+ addopts = -p no:anyio
14
+
15
+ # Markers
16
+ markers =
17
+ browser: Browser integration tests using Playwright