Spaces:
Sleeping
Sleeping
File size: 15,816 Bytes
d86db02 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 | # MediShield AI Document Classification — Implementation Plan
## Workflow Diagram
```
┌─────────────────────────────────┐
│ Uploaded Image │
└──────────────┬──────────────────┘
│
┌──────────────▼──────────────────┐
│ Stage 1: Rules Engine │
│ regex: ^bill_ │
└──────────────┬──────────────────┘
│
┌─────────────────┴──────────────────┐
bill_ match? no match
│ │
┌──────────▼──────────┐ ┌──────────────▼──────────────┐
│ doc_type = "bill" │ │ Stage 2: KYC OCR │
│ method = "rules" │ │ easyocr → keyword regex │
│ ✓ DONE │ └──────────────┬──────────────┘
└─────────────────────┘ │
┌──────────────┴──────────────┐
KYC match? no match
│ │
┌────────────▼────────┐ ┌───────────────▼─────────────┐
│ doc_type = "kyc" │ │ Stage 3: Gemini LLM │
│ method = "ocr" │ │ gemma-4-31b-it │
│ ✓ DONE │ │ → Patient Bills │
└─────────────────────┘ │ → Claim Forms │
│ → Medical Reports │
│ → Prescriptions │
│ → Unknown │
└──────────────────────────────┘
│
┌────────────▼────────────────┐
│ doc_type = "image" │
│ sub_type = <category> │
│ method = "llm" │
│ ✓ DONE │
└─────────────────────────────┘
All stages emit @traceable spans → LangSmith (traces · tokens · latency)
All results served via FastAPI → Drag & Drop UI
Container deployed on Azure Container Apps via GitHub Actions CI/CD
```
## Architecture Overview
```
┌─────────────────────────────────────────────────────┐
│ Frontend UI │
│ Drag & Drop · frontend/index.html │
│ - Batch upload (all files in one POST) │
│ - Concurrent server processing (asyncio.gather) │
│ - Live progress bar + per-file status rows │
│ - Color-coded badges: bill/kyc/image │
└───────────────────┬─────────────────────────────────┘
│ POST /classify (multipart)
▼
┌─────────────────────────────────────────────────────┐
│ FastAPI · src/api.py · Port 8000 │
│ POST /classify · GET /health · GET /metrics │
│ asyncio.gather + run_in_executor (concurrent files) │
└────────────────────┬────────────────────────────────┘
│
┌──────────▼──────────┐
│ src/classifier.py │ (orchestrator)
└──┬──────┬──────┬────┘
│ │ │
rules │ ocr │ llm │
engine │ │ │
▼ ▼ ▼
┌────────────────────────────────────────────────────┐
│ src/monitoring.py — LangSmith @traceable spans │
│ trace_rules_engine · trace_kyc_ocr │
│ trace_llm_classify · trace_classify │
└────────────────────────────────────────────────────┘
│
┌──────────┴──────────┐
▼ ▼
LangSmith Azure Monitor
(traces/tokens) (container logs)
```
## Decision Rules
| Condition | doc_type | method | Sent to LLM? |
|---|---|---|---|
| filename matches `^bill_` (regex) | `bill` | `rules` | No |
| OCR text contains KYC keywords | `kyc` | `ocr` | No |
| Everything else | `image` | `llm` | Yes |
## Final File Layout
```
multimodal-ai/
├── src/
│ ├── rules_engine.py # Step 1 ✅
│ ├── kyc_detector.py # Step 2 ✅
│ ├── llm_classifier.py # Step 3 ✅
│ ├── classifier.py # Step 4 ✅
│ ├── api.py # Step 5 ✅
│ └── monitoring.py # Step 7 ✅
├── frontend/
│ └── index.html # Step 6 ✅
├── tests/
│ ├── test_rules_engine.py # 11 tests ✅
│ ├── test_kyc_detector.py # 21 tests ✅
│ ├── test_llm_classifier.py # 32 tests ✅
│ ├── test_classifier.py # 29 tests ✅
│ ├── test_api.py # 11 tests ✅
│ └── test_monitoring.py # 13 tests ✅ (117 total)
├── infra/
│ ├── deploy.sh # Step 9 ✅ Azure Container Apps
│ └── teardown.sh # ✅
├── .github/workflows/
│ ├── ci.yml # Step 10 ✅ test on every push
│ └── deploy.yml # Step 10 ✅ deploy on merge to main
├── Dockerfile # Step 8 ✅ two-stage build
├── .dockerignore # Step 8 ✅
├── README.md # Step 10 ✅
├── pyproject.toml
└── .env.example
```
---
## Steps
### Phase 1 — Core Classification Engine
- [x] **Step 1 — Rules Engine** (`src/rules_engine.py`)
- Compiled regex `re.compile(r"^bill_")` — case-sensitive, anchored to start of filename
- Strips directory prefix so full paths work (`dataset/bill_x.png`)
- Returns `RulesResult(filename, doc_type, send_to_llm)`
- **Changed from plan:** Used `re.compile` regex instead of `str.startswith()` as requested
- ✅ **11/11 tests passing**
- [x] **Step 2 — KYC Detector** (`src/kyc_detector.py`)
- 11 compiled regex patterns covering Aadhaar, PAN, Passport, Govt of India, DOB, 12-digit Aadhaar number, PAN card format
- easyocr `Reader` is a lazy singleton — loaded once on first use, not at import time
- `reader` is injectable (passed as parameter) so tests never load the real model
- Returns `KYCResult(filename, doc_type, send_to_llm, ocr_text)`
- ✅ **21/21 tests passing**
- [x] **Step 3 — LLM Classifier** (`src/llm_classifier.py`)
- Sends image bytes + structured prompt to `gemma-4-31b-it` via `google-genai`
- Prompt instructs model to return exactly one category name
- `_parse_category()` does case-insensitive match + strips whitespace, falls back to `"Unknown"`
- Captures `input_tokens` and `output_tokens` from `response.usage_metadata`
- `client` is injectable for testing — zero live API calls in test suite
- Returns `LLMResult(filename, doc_type, sub_type, method, input_tokens, output_tokens, raw_response)`
- ✅ **32/32 tests passing**
- [x] **Step 4 — Pipeline Orchestrator** (`src/classifier.py`)
- `classify(filename, image_bytes, ocr_reader, llm_client)` — single document
- `classify_dataset(dataset_dir, ...)` — scans all PNGs in a directory
- Returns `ClassificationResult(filename, doc_type, sub_type, method, latency_ms, input_tokens, output_tokens)`
- Each stage emits a LangSmith trace span (added in Step 7)
- ✅ **29/29 tests passing**
---
### Phase 2 — FastAPI Server
- [x] **Step 5 — API Server** (`src/api.py`)
- `POST /classify` — multipart file upload, returns JSON array
- `GET /health` — liveness probe
- `GET /metrics` — in-memory counters per method/doc_type/token usage
- `GET /docs` — auto Swagger UI
- **Changed from plan:** `asyncio.gather` + `run_in_executor` runs all uploaded files concurrently — `bill_` files return in < 10 ms without waiting behind OCR/LLM calls
- easyocr `Reader` and Gemini `Client` loaded once at startup via FastAPI `lifespan`
- CORS middleware enabled for browser UI
- ✅ **11/11 tests passing** (patched at `src.api.classify`)
---
### Phase 3 — Frontend UI
- [x] **Step 6 — Drag & Drop UI** (`frontend/index.html`)
- Self-contained single HTML file, no external dependencies
- Drag & drop + click-to-browse, deduplicates files by name
- **Changed from plan (sequential → batch):** Sends all files in ONE `POST /classify` — server processes concurrently so `bill_` files don't wait behind slow OCR/LLM calls
- Results table appears immediately with `queued…` rows; fills in as server responds
- Live progress bar + `Processing file N of M` text
- Color-coded badges: bill=blue, kyc=orange, image=green, rules=purple, ocr=red, llm=teal
- All controls (classify, clear, remove buttons, drop zone) disabled during processing
- Summary bar: counts per type + average latency
- Error banner for API failures and unsupported file types
---
### Phase 4 — Monitoring (LangSmith)
- [x] **Step 7 — LangSmith Integration** (`src/monitoring.py`)
- Four `@traceable` functions forming a parent/child span tree:
- `trace_classify` — top-level `chain` span per document
- `trace_rules_engine` — `tool` span for Stage 1
- `trace_kyc_ocr` — `tool` span for Stage 2; records `ocr_text_length` not raw text (PII safety)
- `trace_llm_classify` — `llm` span for Stage 3; records token breakdown
- `record_token_usage()` extracts `input/output/total_tokens` from Gemini `usage_metadata`
- Tracing is a **no-op** when `LANGCHAIN_TRACING_V2` is not set — CI safe
- **Required env vars:**
```
LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=<key>
LANGCHAIN_PROJECT=medishield-classification
```
- ✅ **13/13 tests passing**
---
### Phase 5 — Docker
- [x] **Step 8 — Dockerfile**
- Two-stage build: `uv` builder → `python:3.12-slim` runtime
- Installs OS libs for easyocr/opencv/weasyprint in runtime stage
- **Pre-downloads easyocr models at build time** as `appuser` — container starts in ~10s not 60s
- Runs as non-root `appuser` (with home dir so easyocr can write model cache)
- `HEALTHCHECK` polls `/health` every 30s, 60s start period
- 2 uvicorn workers for concurrency
- **Fix applied during build:** Created home dir for `appuser` and set `EASYOCR_MODULE_PATH` to fix permission error on model cache write
- ✅ **Build verified, `/classify` tested inside container**
---
### Phase 6 — Azure Deployment
- [x] **Step 9 — Azure Container Apps** (`infra/deploy.sh`)
- **Changed from plan:** Azure instead of AWS (simpler setup, no separate load balancer, built-in HTTPS)
- Provisions: Resource Group → ACR → Log Analytics → Container Apps Environment → Container App
- Container App: 0.5 vCPU / 2 GB RAM, min 1 replica, max 3, public HTTPS ingress
- Secrets (`GOOGLE_API_KEY`, `LANGCHAIN_API_KEY`) injected via Container Apps secret references
- `infra/teardown.sh` for full cleanup
- **CI/CD via `.github/workflows/deploy.yml`:**
- Tests gate deploy (deploy only runs if tests pass)
- `az acr build` builds in Azure cloud (no local Docker in CI)
- `az containerapp update` rolling deploy
- Smoke tests live `/health` endpoint post-deploy
- OIDC login (no long-lived secrets in GitHub)
---
### Phase 7 — Documentation
- [x] **Step 10 — README + CI** (`README.md`, `.github/workflows/ci.yml`)
- Professional README with ASCII architecture diagram, workflow diagram, full API reference, setup guide, deployment guide, test matrix, environment variable table
- `ci.yml` runs all 117 tests on every push/PR — no real API keys needed
---
## Build Order Summary
| # | Deliverable | Test Gate | Status |
|---|---|---|---|
| 1 | Rules Engine | `pytest tests/test_rules_engine.py` — 11 passed | ✅ |
| 2 | KYC Detector | `pytest tests/test_kyc_detector.py` — 21 passed | ✅ |
| 3 | LLM Classifier | `pytest tests/test_llm_classifier.py` — 32 passed | ✅ |
| 4 | Orchestrator | `pytest tests/test_classifier.py` — 29 passed | ✅ |
| 5 | FastAPI Server | `pytest tests/test_api.py` — 11 passed + Swagger check | ✅ |
| 6 | Frontend UI | Batch POST, live progress, controls locked during processing | ✅ |
| 7 | LangSmith Monitoring | `pytest tests/test_monitoring.py` — 13 passed | ✅ |
| 8 | Docker | `docker build` + `/classify` tested inside container | ✅ |
| 9 | Azure Deploy | `infra/deploy.sh` + GitHub Actions CI/CD pipeline | ✅ |
| 10 | README + CI | `ci.yml` + `deploy.yml` + `README.md` | ✅ |
**Total: 117 tests · 10 steps · all complete ✅**
## Key Changes vs Original Plan
| Area | Original Plan | What We Actually Built |
|---|---|---|
| Rules matching | `str.startswith("bill_")` | `re.compile(r"^bill_")` regex |
| API concurrency | Sequential file loop | `asyncio.gather` + `run_in_executor` |
| UI upload strategy | One request per file (sequential) | One batch request, server concurrent |
| Cloud provider | AWS ECS Fargate | Azure Container Apps |
| Metrics | OpenTelemetry + CloudWatch | LangSmith + Azure Monitor |
| Docker user | Root | Non-root `appuser` with home dir |
| easyocr models | Downloaded at runtime | Pre-baked into image at build time |
|