Spaces:
Sleeping
Sleeping
| # 🛠️ Work Log — TrafficGuard AI | |
| Running log of what's built. Newest entries at the bottom of each phase. | |
| --- | |
| ## Phase 1 — Foundation & Setup | |
| ### ✅ Backend skeleton (FastAPI) | |
| Project structure scaffolded and byte-compiles clean. | |
| ``` | |
| backend/ | |
| ├── requirements.txt | |
| ├── weights/ # YOLO weights (gitignored) | |
| └── app/ | |
| ├── main.py # FastAPI app, CORS, routers, static evidence mount | |
| ├── config.py # pydantic-settings + storage dir bootstrap | |
| ├── schemas.py # API response models (ViolationOut, AnalysisResult) | |
| ├── routes/ | |
| │ ├── upload.py # POST /api/upload — full analysis pipeline | |
| │ ├── violations.py # GET /api/violations[, /{id}] with filters | |
| │ └── analytics.py # GET /api/analytics/summary, /by-type | |
| ├── models/ | |
| │ ├── preprocessor.py # CLAHE enhance + Laplacian quality score | |
| │ ├── detector.py # YOLOv8 wrapper (lazy-loaded), COCO→category map | |
| │ ├── violation.py # rule engine — triple-riding (IoU) shipped | |
| │ └── ocr.py # EasyOCR plate reader + Indian-format regex | |
| ├── database/ | |
| │ ├── db.py # SQLAlchemy engine/session, init_db | |
| │ └── models.py # Violation table | |
| └── utils/ | |
| ├── annotator.py # draw detection/violation boxes | |
| └── evidence.py # save uploads + annotated evidence | |
| ``` | |
| **Design notes** | |
| - Heavy ML deps (ultralytics, easyocr, cv2) are lazy-loaded so the API boots without weights present. | |
| - Pipeline: upload → preprocess (CLAHE + conditional denoise) → YOLO detect → rule-based violation analysis → annotate → persist to SQLite. | |
| - Phase 1 ships **triple-riding** detection only; helmet/seatbelt/red-light land in Phase 4. | |
| ### ✅ Repo scaffold | |
| - `.gitignore` (Python, node, weights, data artifacts, secrets) | |
| - `data/{uploads,evidence,sample_images,annotations}`, `notebooks/`, `docs/` | |
| ### ✅ Frontend skeleton (Vite + React) | |
| Builds clean (`npm run build`). Editorial light theme — off-white surfaces, | |
| Fraunces serif headings, single teal accent, generous whitespace. No neon. | |
| ``` | |
| frontend/src/ | |
| ├── main.jsx # BrowserRouter bootstrap | |
| ├── App.jsx # routes | |
| ├── api.js # axios client (VITE_API_URL) | |
| ├── index.css # design tokens (palette, fonts, radius) | |
| ├── App.css # component styles | |
| ├── components/ | |
| │ ├── Layout.jsx # sidebar nav + content outlet | |
| │ ├── StatCard.jsx | |
| │ ├── ViolationCard.jsx | |
| │ └── UploadPanel.jsx # drag-and-drop upload | |
| └── pages/ | |
| ├── Dashboard.jsx # stat cards + recent activity | |
| ├── Analyze.jsx # upload → evidence + violations | |
| ├── Violations.jsx # filterable register table | |
| └── Analytics.jsx # recharts bar (by type) | |
| ``` | |
| ### ✅ Environment + end-to-end smoke test | |
| - Root `.venv` (Python 3.13). Installed `backend/requirements.txt` — torch 2.12, | |
| ultralytics 8.3, easyocr, opencv 4.10, fastapi 0.115 all import clean. | |
| - Wired `yolo_weights` to `backend/weights/` (config.py); weights auto-download | |
| there on first inference (`yolov8n.pt`, 6.2 MB ✓). | |
| - Booted `uvicorn app.main:app` and verified the full path: | |
| - `GET /health` → `{"status":"ok"}` | |
| - `POST /api/upload` (blank test frame) → valid `AnalysisResult` | |
| (quality 0, 0 detections, evidence image generated) | |
| - `GET /evidence/<id>.jpg` → `200` | |
| - `GET /api/analytics/summary` → `{"total":0,...}` | |
| - SQLite `trafficguard.db` created at repo root with `violations` table | |
| - No errors/warnings in server log. | |
| > Run: `cd backend && ../.venv/bin/python -m uvicorn app.main:app --reload` | |
| > (must run from `backend/` for the `app.*` imports to resolve) | |
| ### ✅ Cross-platform support (Mac + Windows team) | |
| - `run.sh` (macOS/Linux) + `run.bat` (Windows) — launch the backend via the | |
| project `.venv` regardless of the shell's default `python`/`uvicorn`. | |
| Fixes the anaconda-PATH clash that caused `ModuleNotFoundError: ultralytics`. | |
| - `config.py` SQLite URL now uses `.as_posix()` so it's valid on Windows paths. | |
| - `.gitattributes` normalizes line endings (`.sh` → LF, `.bat` → CRLF, weights/ | |
| images marked binary). | |
| - README setup rewritten with macOS/Linux **and** Windows (PowerShell) steps; | |
| venv lives at repo root. | |
| > Note: run the backend with the **`.venv`**, not anaconda. Either use the | |
| > launcher (`./run.sh` / `run.bat`) or activate the venv first. | |
| ### ✅ Frontend redesign — editorial polish | |
| Full visual pass on the dashboard. Builds + runs clean (Vite HMR on :5173). | |
| - **Design system** (`index.css`): warm paper palette, layered soft shadows, | |
| Fraunces display + Inter body + JetBrains Mono for plates, motion tokens, | |
| `rise` entrance animation, custom scrollbar, subtle radial background. | |
| - **Layout**: refined sidebar (gradient brand mark, active-route accent bar, | |
| nav heading) + sticky blurred topbar with date + "New analysis" CTA + a | |
| **live backend status dot** (polls `/health` every 15s). | |
| - **Dashboard**: icon stat cards (hover-lift, accent variant, trend chips), | |
| a donut "violation mix" with center total + legend, a gradient "get started" | |
| panel, and a recent-activity grid. | |
| - **Analyze**: image-preview on drop, before/after compare (original ↔ | |
| annotated), result metric chips, loader state, feature hint chips. | |
| - **Records**: live search + type filter + count pill, polished sticky-header | |
| table with severity badges and mono plate/IDs. | |
| - **Analytics**: donut + bar charts side by side. | |
| - New shared components: `PageHeader`, `EmptyState`, `StatCard`, `Donut`, | |
| refined `ViolationCard` + `UploadPanel`. | |
| - Responsive: collapses to single column under 940px. | |
| > Note: couldn't auto-screenshot from here — verified via `npm run build` | |
| > (clean) and a dev-server boot (HTTP 200, no errors). View at | |
| > http://localhost:5173. | |
| --- | |
| ## Phase 2 — Image Preprocessing Pipeline | |
| ### ✅ Adaptive preprocessor (`preprocessor.py`) | |
| Assess the frame, then apply **only** the corrections it needs. | |
| - **Quality assessment** → `QualityReport` (0-100): overall `score` plus | |
| `sharpness` (Laplacian var), `brightness` (mean), `contrast` (std). | |
| - **CLAHE** low-light enhancement (L channel) — fires on low brightness / flat contrast. | |
| - **Denoise** (`fastNlMeansDenoisingColored`) — low-light / low overall score. | |
| - **Dehaze** — per-channel percentile contrast stretch (washed-out frames). | |
| - **Deblur** — unsharp mask when Laplacian variance < 100 (motion blur). | |
| - **`normalize()`** — aspect-preserving 640×640 letterbox utility (YOLO does its | |
| own resize, so kept off the main path). | |
| `preprocess()` returns `(image, QualityReport)` and records which fixes ran. | |
| ### ✅ Wired through the stack | |
| - `schemas.py`: new `QualityReport`; `AnalysisResult.quality_score` → `quality`. | |
| - `upload.py`: passes `asdict(report)` into the response. | |
| - Frontend `Analyze.jsx`: quality score chip + **"Auto-corrected:"** chips | |
| listing applied fixes (great demo moment) — styled in `App.css`. | |
| ### ✅ Verified | |
| - Unit test on synthetic dark / hazy / blurry / flat frames — correct | |
| corrections fire for each (e.g. dark → Denoise + CLAHE; blurry → +Sharpen). | |
| - `normalize()` → `(640, 640, 3) uint8`. | |
| - Full serialization path (`pipeline → asdict → AnalysisResult.model_dump_json`) | |
| produces the nested `quality` object. | |
| - Frontend build clean. | |
| --- | |
| ## Phase 3 — Vehicle & Person Detection | |
| ### ✅ Structured detector (`detector.py`) | |
| - `COCO_MAP`: COCO id → `(kind, label, category)` for person, bicycle, car, | |
| motorcycle (→ Two-Wheeler), bus (→ Public Transport), truck (→ Heavy Vehicle). | |
| - Each detection now carries `kind` / `label` / `category` / `bbox` / | |
| `confidence` / `occupants` (conf filter + NMS handled by YOLO). | |
| - **Occupant counting**: `_assign_occupants()` attributes each person to the | |
| vehicle whose box best *contains* them (intersection ÷ person area ≥ 0.2). | |
| Containment (not IoU) is used since a rider is far smaller than the vehicle. | |
| - `summarize()`: road-user counts grouped by category. | |
| ### ✅ Wired through the stack | |
| - `violation.py`: triple-riding now reads `occupants >= 3` from detection | |
| (no more recomputing IoU). | |
| - `annotator.py`: switched `class` → `kind`; shows `xN` occupant count on | |
| vehicles with ≥ 2 riders. | |
| - `schemas.py` + `upload.py`: response gains a `road_users` breakdown. | |
| - Frontend `Analyze.jsx`: **"Road users:"** chip row (count × category). | |
| ### ✅ Verified | |
| - Crafted scene (two-wheeler + 3 overlapping persons + a car) → moto | |
| `occupants = 3`, car `0`, triple-riding fires with correct description. | |
| - `summarize()` groups correctly; full `AnalysisResult` (with `road_users`) | |
| serializes; frontend build clean. | |
| --- | |
| ## Phase 4 — Violation Detection Models ⭐ CORE | |
| ### ✅ Extensible rule framework (`models/rules/` + `violation.py`) | |
| Each rule is a self-contained module exposing `CODE` / `NAME` / `SEVERITY`, | |
| `status()` and `check(scene)`. `violation.py` registers them, runs them all via | |
| `analyze(detections, image)`, and exposes `catalog()`. Add a violation type by | |
| dropping a module in `rules/` and registering it — no other code changes. | |
| `Scene` (rules/base.py) gives each rule `vehicles` / `persons` / `signals`. | |
| | Rule | Status | How it works | | |
| |------|--------|--------------| | |
| | Triple riding | **active** | `occupants >= 3` on a Two-Wheeler | | |
| | Helmet non-compliance | needs-weight | auto-enables when `weights/helmet.pt` exists; runs a helmet YOLO on each rider crop | | |
| | Red-light running | needs-config | HSV signal-colour classify + stop-line; opt-in via `red_light_enforcement` | | |
| | Seatbelt | planned | needs a driver-region classifier | | |
| | Wrong-side driving | planned | needs per-camera lane direction | | |
| | Illegal parking | planned | needs no-parking zone polygons | | |
| ### ✅ Supporting changes | |
| - `detector.py`: traffic lights now detected as `kind="signal"` (COCO 9); | |
| `summarize()` excludes signals. | |
| - `config.py`: `helmet_weights`, `red_light_enforcement`, `stop_line_frac`. | |
| - `GET /api/rules` (+ `RuleInfo` schema) returns the catalogue with statuses. | |
| - Frontend `Dashboard.jsx`: **"Detection coverage"** panel — all 6 rules with | |
| Active / Needs weight / Needs config / Planned tags ("N of 6 active"). | |
| ### ✅ Verified | |
| - `catalog()` → 6 rules with correct statuses. | |
| - Triple riding fires via the coordinator (`analyze(dets, img)`). | |
| - Red-light: with enforcement on + a synthetic red signal, only the vehicle | |
| past the stop line is flagged (the one above the line is not). | |
| - Upload route runs end-to-end in-process; `/api/rules` returns 6; FE build clean. | |
| ### ✅ Helmet rule live with a real model | |
| Wired a trained helmet weight (`helmet.pt`, classes `Plate / WithHelmet / | |
| WithoutHelmet`). Fixes made when it didn't fire: | |
| - Class-name matching generalised (`WithoutHelmet`, `head`, "no … helmet"). | |
| - Runs on the **full frame** (heads sit above the bike box) and attributes each | |
| bare-head box to the nearest two-wheeler by IoU. | |
| - Runs at **imgsz=960** (config `helmet_imgsz`; heads were missed at 640), | |
| conf floor 0.3; dedupes to one helmet violation per vehicle. | |
| - Verified on a real triple-riding photo → TRIPLE_RIDING + HELMET both fire. | |
| - Note: the model also emits `Plate` boxes — reuse for Phase 5 OCR. | |
| ### To enable the model-backed rules | |
| - **Helmet**: drop a helmet-detection YOLO at `backend/weights/helmet.pt` | |
| (e.g. from Roboflow Universe) — the rule flips to *active* automatically. | |
| - **Red-light**: set `red_light_enforcement=true` (and tune `stop_line_frac`) | |
| once the camera's stop line is known. | |
| --- | |
| ## Phase 5 — License Plate Recognition | |
| ### ✅ Plate detection + OCR (`plates.py`, `ocr.py`) | |
| - **Detection** (`PlateService`): uses a dedicated `plate.pt` if present, else | |
| **reuses the helmet model's `Plate` class** (no extra weight needed). Config: | |
| `plate_weights`, `plate_conf`. | |
| - **Preprocess** (`_prep`): upscale small crops to ~96px height + CLAHE contrast. | |
| - **OCR** (`PlateReader`): EasyOCR with an A-Z0-9 **allowlist**, keeps only | |
| fragments with conf ≥ 0.3 (no more garbage), strips the embossed "IND" tag. | |
| Dropped the old blanket O→0/B→8 correction (it corrupted the letter section). | |
| - **Matching**: each plate is attached to the violation whose **vehicle box** | |
| best contains it → `Violation.license_plate` is now populated and shown on | |
| cards / Records table. | |
| - **Evidence**: annotator draws plate boxes (teal) with the OCR text. | |
| ### ✅ Verified | |
| - Synthetic legible plate → reads `MH12AB1234` exactly (OCR path correct). | |
| - Real wide-shot photo: plate region detected but only `02` legible at this | |
| resolution (54×31px) → correctly returns "No plate" instead of guessing. | |
| - False-positive plate on a shop sign is filtered (low conf + outside vehicles). | |
| - Full upload route runs end-to-end with plates wired in. | |
| > ⚠️ Demo note: OCR accuracy depends on plate resolution. Use images where a | |
| > plate is reasonably close/large to show real plate numbers. Distant plates in | |
| > wide shots are too small to read (shown as "No plate", not wrong text). | |
| --- | |
| ## Phase 6 — Evidence Generation & Storage | |
| ### ✅ Evidence provenance + packaging | |
| - `annotator.watermark()`: burns a bottom bar onto evidence — | |
| `TrafficGuard AI | <location> | <timestamp>`. | |
| - `evidence.save_metadata()` / `load_metadata()`: a JSON **evidence package** | |
| sidecar per upload (evidence_id, timestamp, location, image names, quality, | |
| road users, violations). | |
| - `upload.py`: accepts a `location` form field, sets `location` + `timestamp` | |
| on each record, watermarks the image, writes the package JSON. | |
| ### ✅ Review workflow (pending / confirmed / dismissed) | |
| - `PATCH /api/violations/{id}` (`ViolationStatusUpdate`) — confirm/dismiss. | |
| - `GET /api/violations/{id}/evidence` — returns the evidence package. | |
| - Frontend `Records`: status tag + ✓/✗ review buttons (dismissed rows dim), | |
| new **Location** column; `Analyze` has a **camera location** input that flows | |
| through to storage + watermark. | |
| ### ✅ Verified | |
| - Upload with `location="Dadar TT Junction"` → record stores location, evidence | |
| package JSON written with all 8 keys (2 violations inside). | |
| - `PATCH` flips status `pending → confirmed`. | |
| - Backend compiles, full route + status + evidence endpoints run, FE build clean. | |
| --- | |
| ## Phase 7 — Backend API (completed the gaps) | |
| ### ✅ New endpoints | |
| - `POST /api/batch-upload` — runs the pipeline over many images; returns | |
| `BatchResult {processed, total_violations, results[]}`. Pipeline extracted | |
| into shared `process_image()` (used by both `/upload` and `/batch-upload`). | |
| - `GET /api/analytics/trends` — violations per day (`func.date`), oldest first. | |
| - `GET /api/analytics/top-plates?limit=` — top offenders by plate. | |
| Full API surface now: upload, batch-upload, violations (list/get/patch/evidence), | |
| rules, analytics (summary/by-type/trends/top-plates), static `/evidence`. | |
| ### ✅ Surfaced on the Analytics page | |
| - Trend line chart (violations/day) + **Top offenders** ranked list, alongside | |
| the existing donut + bar. `api.js`: `getTrends`, `getTopPlates`. | |
| ### ✅ Verified | |
| - Batch of 2 → processed 2, total_violations 4. | |
| - Trends groups by day correctly; top-plates returns [] here (test images have | |
| unreadable plates — expected). Backend compiles, FE build clean. | |
| --- | |
| ## Frontend polish pass (judge-ready) | |
| - **Animated count-up** on all dashboard stat numbers (`useCountUp` hook + | |
| `StatCard` with numeric value + suffix). | |
| - **Pipeline visualizer** on Analyze (`Pipeline.jsx`): the 5 ML stages | |
| (Enhance → Detect → Classify → Read plates → Evidence) animate while | |
| processing, then show ✓ + real per-stage stats (quality, users, flags, | |
| plates, evidence). Replaces the old static hint chips. | |
| - Verified visually via Playwright screenshots of all pages — dashboard, | |
| analyze (result), analytics, records all render cleanly and cohesively. | |
| (Installed playwright `--no-save`; not in package.json. Chromium cached in | |
| ~/Library, outside the repo.) | |
| ### ⏭️ Next (Phases 9-11) | |
| - Phase 9: test real sample images, edge cases, measure inference time | |
| - Phase 10: deploy (Vercel + Render/Railway) | |
| - Phase 11: pitch deck, demo video, README polish | |
| - Optional: evidence export/download, per-position OCR correction | |