README.md

metadata

title: AVIS - Traffic Violation Intelligence
emoji: 🚦
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
app_port: 7860

AVIS — Automated Violation Intelligence System (Gridlock)

Detects, classifies, and documents traffic violations from single images. Hybrid: deterministic CV detects; a VLM (Gemini, free tier) only verifies ambiguous cases and abstains when a photo can't prove a violation. See docs/DESIGN.md and docs/ROADMAP.md.

Quick start (local dev — zero infra)

python -m venv .venv && .venv\Scripts\activate    # PowerShell: .venv\Scripts\Activate.ps1
pip install -r requirements.txt
cp .env.example .env                              # defaults are fine (SQLite + no VLM)
uvicorn api.main:app --reload                     # open http://127.0.0.1:8000

Upload a traffic image on the dashboard, or POST /images (multipart file). First run downloads the YOLO weights automatically.

Full stack (Postgres + Redis + MinIO)

docker compose up --build

Run the checks

pytest -q            # pure unit tests (rules, scene graph) — no models needed
ruff check . && mypy core/ api/

Status (vertical slice)

Implemented: upload → quality gate (abstain on unusable images) → preprocessing (CLAHE/denoise/low-light) → YOLO detection → evidence graph → traffic-light state → Tier-A rules (helmet, triple-riding) + Tier-C calibration rules (stop-line, red-light, illegal-parking) → confidence fusion + tier-aware routing (auto-confirm / VLM / human / abstain) → license-plate recognition (fast-alpr + Indian-plate regex) → legal mapping → annotated evidence → analytics dashboard + human-review queue (audit trail) + searchable records. Gemini VLM verification is wired and tested.

Tier-C rules need a camera calibration file in configs/<camera_id>.json (sample: configs/cam_demo.json) — pass camera_id on upload. Wrong-side (Tier D) is intentionally inert: a single frame can't prove direction of travel.

Evaluation (python -m eval.run [dataset.json]) — violation-level P/R/F1, a rule-only vs rule+VLM ablation, plate OCR whole/char accuracy, mean latency, and the auto/VLM/human disposition split. Add labelled images under data/eval/ and list them in eval/sample_dataset.json (include clean images with expected: [] to measure false positives).

Architecture

AVIS is built as a production-credible modular monolith. It avoids microservice overhead while providing horizontal scalability via a robust task queue.

1. Three-Tier System

flowchart TD
    %% Styling
    classDef frontend fill:#3b82f6,stroke:#1d4ed8,stroke-width:2px,color:#fff
    classDef api fill:#10b981,stroke:#047857,stroke-width:2px,color:#fff
    classDef db fill:#475569,stroke:#1e293b,stroke-width:2px,color:#fff
    classDef agent fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#fff
    classDef orchestrator fill:#8b5cf6,stroke:#6d28d9,stroke-width:2px,color:#fff

    subgraph Client [Client Tier]
        UI[React Dashboard]:::frontend
        Trace[Observability Trace Logs]:::frontend
    end

    subgraph API_Layer [API & Queue Tier]
        API[FastAPI Backend]:::api
        Redis[(Redis Message Broker)]:::db
    end

    subgraph Storage_Layer [Data & Storage Tier]
        MinIO[(MinIO Object Storage)]:::db
        PG[(PostgreSQL JSONB)]:::db
    end

    subgraph MultiAgent [Multi-Agent Core]
        Worker[Background Worker Process]
        Prep[Quality Gate & Preprocessing]
        YOLO[Vision Agent: YOLO11 Object Detection]:::agent
        ALPR[OCR Agent: Fast-ALPR Engine]:::agent
        Rules[Agent Orchestrator: Rule Engine]:::orchestrator
        Fusion[Agent Orchestrator: Fusion & Router]:::orchestrator
    end

    subgraph External [External AI Services]
        Gemini[VLM Agent: Google Gemini Flash]:::agent
    end

    %% Execution Flow
    UI -->|1. Upload Image| API
    UI -.->|Continuous Poll| Trace
    Trace -.-> API
    
    API -->|2. Store Raw Image| MinIO
    API -->|3. Init Record| PG
    API -->|4. Enqueue Job| Redis
    
    Redis -->|5. Consume| Worker
    Worker -->|Fetch Raw| MinIO
    
    Worker --> Prep
    Prep -->|6. Bbox & Tensors| YOLO
    YOLO -->|7. Crops| ALPR
    ALPR -->|8. Evidence Graph| Rules
    Rules -->|9. Score & Route| Fusion
    
    Fusion -- "10. Ambiguous Case (VLM Route)" --> Gemini
    Gemini -- "Verification Verdict" --> Fusion
    
    Fusion -->|11. Annotated Image| MinIO
    Fusion -->|12. Final Verdict & Log| PG

1. Multi-Agent System Architecture

The system utilizes a specialized multi-agent architecture to maximize both accuracy and performance:

• Specialized Agents: Tasks are divided strictly among specialized models—a Vision Agent (YOLO11) for fast geometric bounding boxes, an OCR Agent (Fast-ALPR) for reading plates, and a VLM Agent (Gemini) for complex visual reasoning.
• Deterministic Orchestration: Instead of relying on a single LLM to guess everything (which leads to hallucinations), an Agent Orchestrator (the Rule Engine) deterministically compiles the outputs of the sub-agents into a verifiable "Evidence Graph."
• Cost & Latency Optimized: The heavy, cloud-based VLM Agent is only invoked for ambiguous edge cases (like verifying red lights). 90% of clear-cut violations are solved instantly by the edge-ready Vision and OCR agents.

2. Three-Tier System

1. Frontend: A React + Vite dashboard. It features a dynamically themed Empty-State Hero dashboard, animated hardware-accelerated gradients, and real-time observability Trace Logs.
2. API & Worker Layer: A FastAPI application handles instant HTTP requests while a Redis-backed queue isolates and processes the heavy computer vision (CV) workloads asynchronously.
3. Data Layer: PostgreSQL stores structured evidence and metadata, while MinIO (S3-compatible) securely stores raw and annotated image assets.

3. Technology Stack

• Core backend: Python 3.11+, FastAPI, Pydantic, SQLModel.
• Computer Vision: Ultralytics YOLO11/YOLOv8 (Object Detection) & fast-alpr (License Plate Recognition).
• Vision-Language Model: Google Gemini Flash via free-tier APIs (used strictly for verification).
• Frontend: React 18, Vite, custom Claude-inspired CSS styling.
• Infrastructure: Docker Compose, PostgreSQL, MinIO, Redis.

All roadmap phases (0–6) are implemented and unit-tested (29 tests).

Helmet detection: if HELMET_WEIGHTS (a fine-tuned YOLO model) is set it's used; otherwise, if LLM_PROVIDER=gemini, the Gemini vision model reads helmet status per rider crop (no model download needed); otherwise helmet is undetermined and routes to VLM/human. The .env here already sets LLM_PROVIDER=gemini, so helmet detection works out of the box once deps are installed.