| # AVIS — Project Status & Context Tracker |
|
|
| > Living document for managing context across sessions. Update it as work lands. |
| > Source of truth for *design* stays `docs/DESIGN.md`; this file tracks *state*. |
| > Last updated: **2026-06-21**. |
|
|
| --- |
|
|
| ## 1. One-line status |
|
|
| End-to-end vertical slice works and is demo-ready: upload → detect → evidence graph → |
| rules → confidence routing → (optional VLM audit) → legal mapping → annotated evidence → |
| **interpretable React dashboard**. Helmet detection now runs on a **local model (zero API |
| calls)**; plate OCR works via **fast-alpr**; all 7 violation types are represented. |
| **Quality gates green:** 49 tests pass, ruff clean, ruff-format clean, mypy clean. |
|
|
| --- |
|
|
| ## 2. Architecture snapshot |
|
|
| ``` |
| upload (api/) → queue (quality gate → preprocess) → pipeline: |
| detect (YOLO11 COCO) core/detect |
| → build Evidence Graph core/graph (single source of truth) |
| → helmet attrs (LOCAL YOLO model) core/detect (no API; was Gemini) |
| → plates (fast-alpr → Gemini fb) core/plates |
| → light state (HSV) core/detect |
| → load calibration zones core/calibration configs/<camera>.json |
| → run rules (pure) core/rules |
| → fuse scores + route + VLM audit core/pipeline (VLM = auditor only) |
| → legal mapping core/legal |
| → hash + annotate + persist core/evidence + core/storage |
| React dashboard (frontend/) ← FastAPI (api/main.py) ← storage (SQLite/Postgres + files/MinIO) |
| ``` |
|
|
| Invariants held: VLM is an auditor (never the detector); only Tier A may auto-confirm; |
| abstain over guess; the Evidence Graph drives all reasoning; rules are pure/deterministic. |
|
|
| --- |
|
|
| ## 3. Violation coverage (vs the problem statement's 7) |
|
|
| | Violation | Tier | Status | How it's decided | |
| |---|---|---|---| |
| | Helmet non-compliance | A | ✅ Working, **offline** | Local 7-class YOLO model → rider helmet status; auto-confirm/VLM-skip | |
| | Triple riding | A | ✅ Working | Count `rides` edges ≥ 3; routed (VLM/human) unless very high fused | |
| | Seatbelt non-compliance | B | ✅ Implemented (gated) | `SEATBELT_CHECK=true` → speculative candidate → VLM verify; dropped if not confirmed | |
| | Stop-line crossing | C | ✅ Working w/ calibration | Vehicle ground-point in `stop_line` zone; never auto-confirm | |
| | Red-light violation | C | ✅ Working w/ calibration | Red light + vehicle past stop-line; candidate | |
| | Illegal parking | C | ✅ Working w/ calibration | Vehicle in `no_parking` zone; candidate | |
| | Wrong-side driving | D | ⚪ Inert by design | A single frame can't prove direction → abstains (needs video) | |
|
|
| Plate OCR (all types): ✅ fast-alpr (ONNX) + Indian-plate regex; Gemini fallback when empty. |
|
|
| --- |
|
|
| ## 4. Problem-statement task coverage |
|
|
| | Task | Status | Notes | |
| |---|---|---| |
| | Image preprocessing (low-light/blur/etc.) | ✅ | CLAHE + denoise + gamma; quality gate abstains on too-dark/blurry/over-exposed | |
| | Vehicle + road-user detection & classification | ✅ | YOLO11 COCO; rider/driver/pedestrian roles in the graph | |
| | Violation detection (7 types) | ✅ 6 active + 1 inert | see §3 | |
| | Violation classification + confidence | ✅ | per-source scores (detection/rule/attribute/vlm) → fused; tier-aware routing | |
| | License-plate detection + OCR | ✅ | fast-alpr + regex; Gemini fallback | |
| | Evidence generation (annotated + metadata + timestamps) | ✅ | annotated copy, SHA-256 hash, audit trail, created_at | |
| | Analytics & reporting (trends/search/summary) | ✅ | `/analytics`, charts, review queue, search filters | |
| | Performance evaluation (P/R/F1, etc.) | ⚙️ harness ready | `eval/` computes P/R/F1, ablation, OCR acc, latency — **needs labelled images in `data/eval/`** | |
| | Efficiency / scalability | ✅ design | modular monolith; Redis-worker + Postgres + MinIO documented scale-out | |
| |
| --- |
| |
| ## 5. Task board |
| |
| **Done (this iteration)** |
| - [x] Local helmet model wired in (`models/helmet/best.pt`, 7-class) — helmet detection needs **zero** API calls. |
| - [x] Plate OCR fixed (root cause: missing `onnxruntime`) + per-char-confidence bug + Gemini fallback chain. |
| - [x] Seatbelt (Tier B) rule + speculative drop-on-negative adjudication (off by default). |
| - [x] Interpretability API: `GET /violations/{id}`, image dims on `/images/{id}`, extended `/runtime`. |
| - [x] New **React + Vite + Chart.js** dashboard with full interpretability + e-challan view. |
| - [x] Tests: 33 → **49**; ruff/format/mypy all clean. |
| |
| **Backlog / next** |
| - [ ] Add a labelled `data/eval/` set and publish real P/R/F1 + ablation numbers. |
| - [ ] Persist the Evidence Graph (per image) to enable client-side bbox overlays in the detail view. |
| - [ ] Optional: seatbelt via a local classifier (avoid VLM quota) instead of VLM-only. |
| - [ ] Optional: ChromaDB legal-RAG bonus (static table stays load-bearing). |
| - [x] Docker: multi-stage build compiles `frontend/dist` in a node stage (one-command deploy). |
| |
| --- |
| |
| ## 6. Change log |
| |
| - **2026-06-21** |
| - Helmet: replaced per-rider Gemini classification with a local YOLO11 model run once on |
| the full image; matches helmet boxes to riders by containment (head boxes have tiny IoU |
| but high containment) and augments riders COCO missed. Fixes the free-tier rate-limit flood. |
| - Plates: added `onnxruntime` (the missing piece that made fast-alpr silently return nothing); |
| fixed `confidence`-is-a-list crash; added `GeminiPlateRecognizer` + `ChainPlateRecognizer`. |
| - Seatbelt: new Tier-B rule (gated by `SEATBELT_CHECK`); `Candidate.speculative` + adjudication |
| that **drops** unconfirmed guesses instead of flooding human review. |
| - API: `GET /violations/{id}`, `width`/`height` on `GET /images/{id}`, `vlm_enabled` + thresholds on `/runtime`. |
| - Frontend: full React rewrite (interpretable cards, confidence breakdown, route explainer, |
| e-challan, audit trail, evidence hash, charts, review, search). |
| - Quality: brought the whole repo to ruff/format/mypy clean (cleared pre-existing debt). |
| - Config fix: `.env` had `GEMINI_MODEL=gGemma-4-31B` (invalid id) → `gemma-3-27b-it` |
| (vision-capable; Gemma 3 free tier ~30 RPM / ~15,000 RPD, much higher than Flash). |
| |
| --- |
|
|
| ## 7. Issue / error log (root causes) |
|
|
| | Symptom | Root cause | Fix | |
| |---|---|---| |
| | "VLM unavailable" flooding human review on busy images | ~14 Gemini calls/image (helmet ×N + verify ×N) > 10 req/min free limit | Local helmet model (0 calls) + helmet candidates `pre_verified` (skip 2nd call) + retry/backoff | |
| | Number plate never shown | `fast_alpr` installed but `onnxruntime` missing → ONNX models couldn't run → silent "no plate" | `pip install onnxruntime` (added to requirements) | |
| | fast-alpr crash `float() ... not 'list'` | OCR `confidence` is a per-character list | `_as_conf()` collapses list → mean | |
| | Dashboard "too raw" | vanilla HTML dumped enums | React dashboard with friendly labels, badges, breakdowns, e-challan | |
|
|
| --- |
|
|
| ## 8. Known honest limitations |
|
|
| - **Seatbelt (Tier B)** is genuinely unreliable from a traffic cam; it never auto-confirms and |
| is dropped unless the VLM can substantiate it. Off by default to protect quota. |
| - **Wrong-side (Tier D)** is inert — direction needs video; a single frame abstains. |
| - **Tier C** (stop-line/red-light/parking) needs per-camera calibration (`configs/<camera>.json`); |
| without it, those rules simply don't fire (no fabrication). |
| - **Eval numbers** require a labelled `data/eval/` set — not shipped, so we publish no fake metrics. |
| - **Gemini free tier** (~1,500/day, ~10/min): plate fallback + seatbelt are the only paths that |
| can spend it; both are sparing/optional. |
|
|
| --- |
|
|
| ## 9. Setup & run (quick reference) |
|
|
| ```powershell |
| # deps (one-time) |
| venv\Scripts\python.exe -m pip install -r requirements.txt # incl. onnxruntime |
| cd frontend ; npm install ; npm run build ; cd .. # builds frontend/dist |
| |
| # run (serves the built React app at http://127.0.0.1:8000) |
| venv\Scripts\python.exe -m uvicorn api.main:app --reload |
| |
| # frontend dev mode (hot reload, proxies API to :8000) |
| cd frontend ; npm run dev |
| |
| # quality gates |
| venv\Scripts\python.exe -m pytest -q |
| venv\Scripts\python.exe -m ruff check . ; venv\Scripts\python.exe -m ruff format --check . |
| venv\Scripts\python.exe -m mypy core/ api/ |
| ``` |
|
|
| Key env (`.env`): `HELMET_WEIGHTS=models/helmet/best.pt`, `PLATE_PROVIDER=fastalpr`, |
| `LLM_PROVIDER=gemini`, `GEMINI_MODEL=gemini-2.5-flash`, `SEATBELT_CHECK=false`. |
| **Security:** rotate `GEMINI_API_KEY` (it was shared in chat); `.env` is gitignored. |
|
|
