# AVIS Improvement & Implementation Plan (Free-Tier Optimized) This document provides a highly detailed, comprehensive roadmap for implementing advanced features in AVIS while strictly adhering to the API limits of Groq (14.4K requests/day) and Gemini (500 requests/day). --- ## 1. The Dual-AI Processing Engine To maximize accuracy without hitting rate limits, AVIS will split its cognitive load between Groq (Text) and Gemini (Vision). ### A. Groq LPU: OCR Error Correction & Legal Formatting - **How it works (Technical)**: When a license plate is detected, generic OCR tools often make mistakes (e.g., reading "0" as "O", or merging characters). We will pass the raw OCR string and a strict prompt to the `llama-3.1-8b-instant` model via the Groq API. The prompt will enforce standard Indian Plate Regex (`^[A-Z]{2}[0-9]{2}[A-Z]{1,2}[0-9]{4}$`). Groq will also be tasked with generating a 2-sentence legal summary of the violation based on the JSON evidence graph. - **How it is used (Practical)**: The user uploads an image. Within 200ms, the system detects a blurry plate reading "DL8C AA123". Groq instantly corrects this to "DL8CAA0123". The user never sees the error. When they click "Generate Challan," Groq instantly writes the legal text: *"Vehicle DL8CAA0123 was observed with a rider failing to wear a helmet..."* ### B. Gemini VLM: The Vision Edge-Case Resolver - **How it works (Technical)**: YOLO handles 90% of geometric detections (e.g., drawing a box around a helmet). If YOLO confidence is between 40% and 75% (ambiguous), the backend triggers a fallback. It sends the *cropped image* of the rider to the `Gemini 3.1 Flash Lite` API, asking a simple binary question: "Is this person wearing a helmet? True or False." - **How it is used (Practical)**: An Admin uploads a tricky photo where a rider is wearing a cap instead of a helmet. YOLO is confused (60% confidence). The dashboard shows a loading spinner reading: *"Routing to Gemini VLM for Verification"*. Three seconds later, it confirms a violation. This prevents the Admin from having to manually review false positives. --- ## 2. Advanced Features (The AVIS Edge) ### A. Geo-Risk Predictive Analytics (Map Grounding) - **How it works (Technical)**: We integrate Google's Map Grounding API (included in Gemini's free tier). When a violation is stored in PostgreSQL, it must include a `location_name` (e.g., "Connaught Place"). A scheduled Redis background task queries the Map Grounding API to assess the traffic density and historical accident risk of that specific intersection based on live map data. - **How it is used (Practical)**: The Admin logs into the dashboard and clicks the **Analytics** tab. Alongside standard bar charts, they see a "Geo-Risk Map". Intersections are color-coded. Clicking a red zone shows: *"High Risk: Connaught Place. Gemini Map Grounding indicates high pedestrian traffic. Recommended action: Deploy physical officers."* ### B. ReportLab E-Challan Generation (End-to-End Enforcement) - **How it works (Technical)**: We implement the `reportlab` Python library in the FastAPI backend. Upon clicking a button, the system generates a PDF template. It embeds the original evidence photo, the zoomed-in plate crop, the Groq-generated legal text, and a QR code linking to a dummy payment gateway. - **How it is used (Practical)**: In the Admin dashboard, an officer reviews a verified violation. They click the new **"Issue E-Challan"** button. The system immediately downloads a beautifully formatted, official-looking PDF ticket that can be emailed or printed. --- ## 3. Catching up to RoadX (Core Upgrades) ### A. Video & RTSP Processing via ByteTrack - **How it works (Technical)**: Instead of just running YOLO on single frames, we implement `ultralytics` native tracking (`model.track()`). ByteTrack assigns a unique ID (e.g., `id: 4`) to a motorcycle. By tracking `id: 4` across 30 frames, the system calculates a trajectory. To save Gemini limits, we extract the *single frame* where the bounding box is largest/clearest, and run our standard single-image pipeline on that frame alone. - **How it is used (Practical)**: The user drags an `.mp4` dashcam video into the upload box. The system processes it in the background. Instead of returning 30 separate violations for the same car, it groups them into a single incident card: *"Motorcycle #4 detected driving Wrong-Way for 6 seconds."* ### B. Custom Plate Pipeline (`Plate.pt`) - **How it works (Technical)**: Generic ALPR struggles in India. We will replace it by training (or downloading) a custom YOLOv8 model explicitly trained on Indian license plates. We use this model to extract a perfect bounding box crop of the plate, then pass that high-res crop to `EasyOCR`. - **How it is used (Practical)**: The system achieves near-perfect plate recognition even at sharp angles or in low light, meaning the Admin rarely has to manually correct plate numbers in the dashboard. --- ## 4. Demo Simulation Scenario: "Operation Gridlock" To showcase this system in a demo video, we will script a flawless run-through using pre-seeded SQLite data. 1. **The Hero Start**: The video starts on the sleek, animated Empty-State dashboard. 2. **The Fast OCR (Groq)**: The user uploads a photo of a speeding car. YOLO draws the box, and we show the terminal logs where `fast-alpr` fails, but Groq instantly corrects the plate to `KA01AB1234` in under 1 second. 3. **The Smart Fallback (Gemini)**: The user uploads a photo of three people on a bike (Triple Riding). One person is partially hidden. The UI pauses, shows *"Consulting Gemini Vision..."*, and successfully confirms 3 riders. 4. **The PDF Generation**: The user clicks "Issue Challan". The PDF pops open on screen, showing the evidence photo, the plate crop, and the Groq-authored legal summary. 5. **The Map Grounding Reveal**: The user clicks the Analytics page to reveal the Gemini-powered Geo-Risk Heatmap of New Delhi.