Hugging Face's logo

Spaces:
X2-0
/
AVIS
Sleeping

App Files Community
AVIS / improvement_plan.md
X2-0's picture
X2-0
HF Clean Deploy
1c0c94d
preview code
|
Raw
History Blame Contribute Delete
5.96 kB

AVIS Improvement & Implementation Plan (Free-Tier Optimized)

This document provides a highly detailed, comprehensive roadmap for implementing advanced features in AVIS while strictly adhering to the API limits of Groq (14.4K requests/day) and Gemini (500 requests/day).

1. The Dual-AI Processing Engine

To maximize accuracy without hitting rate limits, AVIS will split its cognitive load between Groq (Text) and Gemini (Vision).

A. Groq LPU: OCR Error Correction & Legal Formatting

  • How it works (Technical): When a license plate is detected, generic OCR tools often make mistakes (e.g., reading "0" as "O", or merging characters). We will pass the raw OCR string and a strict prompt to the llama-3.1-8b-instant model via the Groq API. The prompt will enforce standard Indian Plate Regex (^[A-Z]{2}[0-9]{2}[A-Z]{1,2}[0-9]{4}$). Groq will also be tasked with generating a 2-sentence legal summary of the violation based on the JSON evidence graph.
  • How it is used (Practical): The user uploads an image. Within 200ms, the system detects a blurry plate reading "DL8C AA123". Groq instantly corrects this to "DL8CAA0123". The user never sees the error. When they click "Generate Challan," Groq instantly writes the legal text: "Vehicle DL8CAA0123 was observed with a rider failing to wear a helmet..."

B. Gemini VLM: The Vision Edge-Case Resolver

  • How it works (Technical): YOLO handles 90% of geometric detections (e.g., drawing a box around a helmet). If YOLO confidence is between 40% and 75% (ambiguous), the backend triggers a fallback. It sends the cropped image of the rider to the Gemini 3.1 Flash Lite API, asking a simple binary question: "Is this person wearing a helmet? True or False."
  • How it is used (Practical): An Admin uploads a tricky photo where a rider is wearing a cap instead of a helmet. YOLO is confused (60% confidence). The dashboard shows a loading spinner reading: "Routing to Gemini VLM for Verification". Three seconds later, it confirms a violation. This prevents the Admin from having to manually review false positives.

2. Advanced Features (The AVIS Edge)

A. Geo-Risk Predictive Analytics (Map Grounding)

  • How it works (Technical): We integrate Google's Map Grounding API (included in Gemini's free tier). When a violation is stored in PostgreSQL, it must include a location_name (e.g., "Connaught Place"). A scheduled Redis background task queries the Map Grounding API to assess the traffic density and historical accident risk of that specific intersection based on live map data.
  • How it is used (Practical): The Admin logs into the dashboard and clicks the Analytics tab. Alongside standard bar charts, they see a "Geo-Risk Map". Intersections are color-coded. Clicking a red zone shows: "High Risk: Connaught Place. Gemini Map Grounding indicates high pedestrian traffic. Recommended action: Deploy physical officers."

B. ReportLab E-Challan Generation (End-to-End Enforcement)

  • How it works (Technical): We implement the reportlab Python library in the FastAPI backend. Upon clicking a button, the system generates a PDF template. It embeds the original evidence photo, the zoomed-in plate crop, the Groq-generated legal text, and a QR code linking to a dummy payment gateway.
  • How it is used (Practical): In the Admin dashboard, an officer reviews a verified violation. They click the new "Issue E-Challan" button. The system immediately downloads a beautifully formatted, official-looking PDF ticket that can be emailed or printed.

3. Catching up to RoadX (Core Upgrades)

A. Video & RTSP Processing via ByteTrack

  • How it works (Technical): Instead of just running YOLO on single frames, we implement ultralytics native tracking (model.track()). ByteTrack assigns a unique ID (e.g., id: 4) to a motorcycle. By tracking id: 4 across 30 frames, the system calculates a trajectory. To save Gemini limits, we extract the single frame where the bounding box is largest/clearest, and run our standard single-image pipeline on that frame alone.
  • How it is used (Practical): The user drags an .mp4 dashcam video into the upload box. The system processes it in the background. Instead of returning 30 separate violations for the same car, it groups them into a single incident card: "Motorcycle #4 detected driving Wrong-Way for 6 seconds."

B. Custom Plate Pipeline (Plate.pt)

  • How it works (Technical): Generic ALPR struggles in India. We will replace it by training (or downloading) a custom YOLOv8 model explicitly trained on Indian license plates. We use this model to extract a perfect bounding box crop of the plate, then pass that high-res crop to EasyOCR.
  • How it is used (Practical): The system achieves near-perfect plate recognition even at sharp angles or in low light, meaning the Admin rarely has to manually correct plate numbers in the dashboard.

4. Demo Simulation Scenario: "Operation Gridlock"

To showcase this system in a demo video, we will script a flawless run-through using pre-seeded SQLite data.

  1. The Hero Start: The video starts on the sleek, animated Empty-State dashboard.
  2. The Fast OCR (Groq): The user uploads a photo of a speeding car. YOLO draws the box, and we show the terminal logs where fast-alpr fails, but Groq instantly corrects the plate to KA01AB1234 in under 1 second.
  3. The Smart Fallback (Gemini): The user uploads a photo of three people on a bike (Triple Riding). One person is partially hidden. The UI pauses, shows "Consulting Gemini Vision...", and successfully confirms 3 riders.
  4. The PDF Generation: The user clicks "Issue Challan". The PDF pops open on screen, showing the evidence photo, the plate crop, and the Groq-authored legal summary.
  5. The Map Grounding Reveal: The user clicks the Analytics page to reveal the Gemini-powered Geo-Risk Heatmap of New Delhi.