| --- |
| title: AVIS - Traffic Violation Intelligence |
| emoji: 🚦 |
| colorFrom: blue |
| colorTo: indigo |
| sdk: docker |
| pinned: false |
| app_port: 7860 |
| --- |
| |
| # AVIS — Automated Violation Intelligence System (Gridlock) |
|
|
| Detects, classifies, and documents traffic violations from **single** images. Hybrid: |
| deterministic CV detects; a VLM (Gemini, free tier) only *verifies* ambiguous cases and |
| *abstains* when a photo can't prove a violation. See [`docs/DESIGN.md`](docs/DESIGN.md) |
| and [`docs/ROADMAP.md`](docs/ROADMAP.md). |
|
|
| ## Quick start (local dev — zero infra) |
|
|
| ```bash |
| python -m venv .venv && .venv\Scripts\activate # PowerShell: .venv\Scripts\Activate.ps1 |
| pip install -r requirements.txt |
| cp .env.example .env # defaults are fine (SQLite + no VLM) |
| uvicorn api.main:app --reload # open http://127.0.0.1:8000 |
| ``` |
|
|
| Upload a traffic image on the dashboard, or `POST /images` (multipart `file`). First run |
| downloads the YOLO weights automatically. |
|
|
| ## Full stack (Postgres + Redis + MinIO) |
|
|
| ```bash |
| docker compose up --build |
| ``` |
|
|
| ## Run the checks |
|
|
| ```bash |
| pytest -q # pure unit tests (rules, scene graph) — no models needed |
| ruff check . && mypy core/ api/ |
| ``` |
|
|
| ## Status (vertical slice) |
|
|
| Implemented: upload → **quality gate** (abstain on unusable images) → **preprocessing** |
| (CLAHE/denoise/low-light) → YOLO detection → evidence graph → **traffic-light state** → |
| **Tier-A** rules (helmet, triple-riding) + **Tier-C** calibration rules (stop-line, |
| red-light, illegal-parking) → confidence fusion + tier-aware routing (auto-confirm / VLM / |
| human / abstain) → **license-plate recognition** (fast-alpr + Indian-plate regex) → legal |
| mapping → annotated evidence → **analytics dashboard + human-review queue (audit trail) + |
| searchable records**. Gemini VLM verification is wired and tested. |
|
|
| Tier-C rules need a camera calibration file in `configs/<camera_id>.json` (sample: |
| `configs/cam_demo.json`) — pass `camera_id` on upload. Wrong-side (Tier D) is intentionally |
| inert: a single frame can't prove direction of travel. |
|
|
| **Evaluation** (`python -m eval.run [dataset.json]`) — violation-level P/R/F1, a |
| **rule-only vs rule+VLM ablation**, plate OCR whole/char accuracy, mean latency, and the |
| auto/VLM/human disposition split. Add labelled images under `data/eval/` and list them in |
| `eval/sample_dataset.json` (include clean images with `expected: []` to measure false |
| positives). |
|
|
| ## Architecture |
|
|
| AVIS is built as a **production-credible modular monolith**. It avoids microservice overhead while providing horizontal scalability via a robust task queue. |
|
|
| ### 1. Three-Tier System |
|
|
| ```mermaid |
| flowchart TD |
| %% Styling |
| classDef frontend fill:#3b82f6,stroke:#1d4ed8,stroke-width:2px,color:#fff |
| classDef api fill:#10b981,stroke:#047857,stroke-width:2px,color:#fff |
| classDef db fill:#475569,stroke:#1e293b,stroke-width:2px,color:#fff |
| classDef agent fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#fff |
| classDef orchestrator fill:#8b5cf6,stroke:#6d28d9,stroke-width:2px,color:#fff |
| |
| subgraph Client [Client Tier] |
| UI[React Dashboard]:::frontend |
| Trace[Observability Trace Logs]:::frontend |
| end |
| |
| subgraph API_Layer [API & Queue Tier] |
| API[FastAPI Backend]:::api |
| Redis[(Redis Message Broker)]:::db |
| end |
| |
| subgraph Storage_Layer [Data & Storage Tier] |
| MinIO[(MinIO Object Storage)]:::db |
| PG[(PostgreSQL JSONB)]:::db |
| end |
| |
| subgraph MultiAgent [Multi-Agent Core] |
| Worker[Background Worker Process] |
| Prep[Quality Gate & Preprocessing] |
| YOLO[Vision Agent: YOLO11 Object Detection]:::agent |
| ALPR[OCR Agent: Fast-ALPR Engine]:::agent |
| Rules[Agent Orchestrator: Rule Engine]:::orchestrator |
| Fusion[Agent Orchestrator: Fusion & Router]:::orchestrator |
| end |
| |
| subgraph External [External AI Services] |
| Gemini[VLM Agent: Google Gemini Flash]:::agent |
| end |
| |
| %% Execution Flow |
| UI -->|1. Upload Image| API |
| UI -.->|Continuous Poll| Trace |
| Trace -.-> API |
| |
| API -->|2. Store Raw Image| MinIO |
| API -->|3. Init Record| PG |
| API -->|4. Enqueue Job| Redis |
| |
| Redis -->|5. Consume| Worker |
| Worker -->|Fetch Raw| MinIO |
| |
| Worker --> Prep |
| Prep -->|6. Bbox & Tensors| YOLO |
| YOLO -->|7. Crops| ALPR |
| ALPR -->|8. Evidence Graph| Rules |
| Rules -->|9. Score & Route| Fusion |
| |
| Fusion -- "10. Ambiguous Case (VLM Route)" --> Gemini |
| Gemini -- "Verification Verdict" --> Fusion |
| |
| Fusion -->|11. Annotated Image| MinIO |
| Fusion -->|12. Final Verdict & Log| PG |
| ``` |
|
|
| ### 1. Multi-Agent System Architecture |
| The system utilizes a specialized multi-agent architecture to maximize both accuracy and performance: |
| - **Specialized Agents**: Tasks are divided strictly among specialized models—a **Vision Agent** (YOLO11) for fast geometric bounding boxes, an **OCR Agent** (Fast-ALPR) for reading plates, and a **VLM Agent** (Gemini) for complex visual reasoning. |
| - **Deterministic Orchestration**: Instead of relying on a single LLM to guess everything (which leads to hallucinations), an **Agent Orchestrator** (the Rule Engine) deterministically compiles the outputs of the sub-agents into a verifiable "Evidence Graph." |
| - **Cost & Latency Optimized**: The heavy, cloud-based VLM Agent is only invoked for ambiguous edge cases (like verifying red lights). 90% of clear-cut violations are solved instantly by the edge-ready Vision and OCR agents. |
|
|
| ### 2. Three-Tier System |
| 1. **Frontend**: A React + Vite dashboard. It features a dynamically themed Empty-State Hero dashboard, animated hardware-accelerated gradients, and real-time observability Trace Logs. |
| 2. **API & Worker Layer**: A FastAPI application handles instant HTTP requests while a Redis-backed queue isolates and processes the heavy computer vision (CV) workloads asynchronously. |
| 3. **Data Layer**: PostgreSQL stores structured evidence and metadata, while MinIO (S3-compatible) securely stores raw and annotated image assets. |
|
|
| ### 3. Technology Stack |
| - **Core backend**: Python 3.11+, FastAPI, Pydantic, SQLModel. |
| - **Computer Vision**: Ultralytics YOLO11/YOLOv8 (Object Detection) & `fast-alpr` (License Plate Recognition). |
| - **Vision-Language Model**: Google Gemini Flash via free-tier APIs (used strictly for verification). |
| - **Frontend**: React 18, Vite, custom Claude-inspired CSS styling. |
| - **Infrastructure**: Docker Compose, PostgreSQL, MinIO, Redis. |
|
|
| All roadmap phases (0–6) are implemented and unit-tested (29 tests). |
|
|
| > Helmet detection: if `HELMET_WEIGHTS` (a fine-tuned YOLO model) is set it's used; |
| > otherwise, if `LLM_PROVIDER=gemini`, the Gemini vision model reads helmet status per |
| > rider crop (no model download needed); otherwise helmet is undetermined and routes to |
| > VLM/human. The `.env` here already sets `LLM_PROVIDER=gemini`, so helmet detection works |
| > out of the box once deps are installed. |
| |
