| --- |
| title: Gridlock Traffic Violation API |
| emoji: 🚦 |
| colorFrom: red |
| colorTo: blue |
| sdk: gradio |
| python_version: "3.10" |
| app_file: app.py |
| pinned: false |
| --- |
| # AID 728 — Traffic Rule Violation Detection |
|
|
| **IIIT Bangalore** |
|
|
| Detects traffic rule violations involving two-wheelers from single RGB street-camera images. Identifies **helmet violations**, **over-riding (>2 riders on one bike)**, and extracts the **license plate text** of every violating vehicle. |
|
|
| --- |
|
|
| ## Submission Files |
|
|
| ``` |
| final_submission/ |
| ├── solution.py # Core detection pipeline (TrafficViolationDetector class) |
| ├── requirements.txt # All Python dependencies |
| ├── README.md # This file |
| └── models/ # All model weights (bundled, fully offline) |
| ├── yolov8s.pt # COCO primary detector (21.54 MB) |
| ├── stage1_best.pt # Custom two-wheeler detector (21.49 MB) |
| ├── helmet_v11.pt # Helmet classifier (5.22 MB) |
| ├── license.pt # License plate localiser (42.77 MB) |
| ├── FSRCNN_x3.pb # Super-resolution for plates (0.04 MB) |
| ├── depth_anything_v2/ # Depth-Anything V2 Small (HF) (47.31 MB fp16) |
| └── paddleocr/ # Bundled PaddleOCR models |
| └── official_models/ |
| └── ... |
| |
| The pipeline also uses the `inference_sdk` to query the Roboflow API for: |
| - **Wrong-way driving detection** (`wrong-way-driving-detection-gqdmg/1`) |
| - **Seatbelt classification** (`seat-belt-detection-udcfg/5`) |
| |
| Total model size: 194.59 MB (limit: 250 MB) |
| ``` |
|
|
| --- |
|
|
| ## Quick Start |
|
|
| ### Install dependencies |
| ```bash |
| pip install -r requirements.txt |
| ``` |
|
|
| ### Run inference |
| ```python |
| from solution import TrafficViolationDetector |
| |
| detector = TrafficViolationDetector(model_dir="./models") |
| result = detector.predict("path/to/image.jpg") |
| print(result) |
| ``` |
|
|
| ### Output format |
| ```json |
| { |
| "violations": [ |
| { |
| "vehicle_type": "two_wheeler", |
| "num_riders": 2, |
| "helmet_violations": 1, |
| "wrong_way": false, |
| "license_plate": "DL 7S AF 8144" |
| }, |
| { |
| "vehicle_type": "four_wheeler", |
| "seatbelt_violations": 1, |
| "wrong_way": true, |
| "license_plate": "MH 12 AB 1234" |
| } |
| ] |
| } |
| ``` |
|
|
| - One entry per **violating** two-wheeler only |
| - `violations` is an empty list `[]` if no violations are found |
| - `license_plate` is `"UNKNOWN"` when the plate cannot be read |
| - `num_riders` counts riders per bike; `helmet_violations` counts those without a helmet |
|
|
| --- |
|
|
| ## Pipeline Architecture |
|
|
| The pipeline runs in 7 sequential stages per image: |
|
|
| ``` |
| Input Image |
| │ |
| ▼ |
| ┌─────────────────────────────────────────────────────────────────┐ |
| │ Stage 1 — Primary Detection (yolov8s.pt, COCO) │ |
| │ Detects: persons (cls 0), motorcycles (cls 3) │ |
| └────────────────────────┬────────────────────────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 2 — Supplemental Bike Detection │ |
| │ (stage1_best.pt — custom trained) │ |
| │ Merged with Stage 1 bikes via NMS │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 3 — Monocular Depth Estimation │ |
| │ (Depth-Anything V2 Small, fp16 stored) │ |
| │ Produces normalised depth map [0,1] │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 4 — Person → Bike Association │ |
| │ Criteria: IoU overlap + column align │ |
| │ + depth proximity check │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌──────────▼──────────┐ |
| │ Per-bike loop │ |
| └──────────┬──────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 5 — Helmet Classification │ |
| │ (helmet_v11.pt — YOLOv11 custom) │ |
| │ Crops top 45% of each rider bbox │ |
| │ (head region), runs cls 0=helmet │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 6 — Wrong Way Detection (API) │ |
| │ (wrong-way-driving-detection-gqdmg/1) │ |
| │ Flags vehicle bounding boxes that │ |
| │ overlap with 'wrong-side' detections │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 7 — Seatbelt Detection (API) │ |
| │ (seat-belt-detection-udcfg/5) │ |
| │ Runs only on four-wheeler crops │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 8 — License Plate Localisation │ |
| │ (license.pt — YOLO custom) │ |
| │ Runs on violating vehicles │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ┌────────────────────▼────────────────────┐ |
| │ Stage 9 — OCR (PaddleOCR 3.5.0) │ |
| │ FSRCNN x3 super-resolution → CLAHE │ |
| │ sharpening → PP-OCRv5 mobile det+rec │ |
| │ Text cleaned: uppercase alphanumeric │ |
| └────────────────────┬────────────────────┘ |
| │ |
| ▼ |
| Output: violations list |
| ``` |
|
|
| ### Violation Logic |
| - A bike is flagged as a **violation** if: |
| - `num_riders >= 3` (over-riding), **OR** |
| - `helmet_violations > 0` (at least one rider without a helmet) |
| - Only violating bikes appear in the output list |
|
|
| --- |
|
|
| ## Model Details |
|
|
| ### `yolov8s.pt` — COCO Primary Detector |
| - **Type**: YOLOv8 Small, pretrained on COCO |
| - **Used for**: Detecting persons (class 0) and motorcycles (class 3) |
| - **Confidence**: 0.30, IoU: 0.45 |
|
|
| ### `stage1_best.pt` — Custom Two-Wheeler Detector |
| - **Type**: YOLOv8-based, custom trained |
| - **Used for**: Supplementing COCO detections with domain-specific two-wheeler types (scooters, three-wheelers, etc. that COCO misses) |
| - **Merge**: Combined with COCO bike boxes via IoU-based NMS (threshold 0.45) |
| - **Augmented inference** (`augment=True`) for improved recall |
| |
| ### `depth_anything_v2/` — Monocular Depth Estimation |
| - **Type**: Depth-Anything V2 Small (Hugging Face Transformers) |
| - **Used for**: Filtering out background pedestrians that share column overlap with a detected bike but are at a different depth plane |
| - **Storage**: fp16 safetensors on disk (47.3 MB vs 94.6 MB fp32) — loaded as fp32 at runtime for CPU inference speed |
| - **Output**: Normalised depth map [0, 1] resized to match the input image |
| |
| ### `helmet_v11.pt` — Helmet Classifier |
| - **Type**: YOLOv11-based, custom trained on merged dataset |
| - **Training data**: 4 merged Kaggle datasets (andrewmvd, aneesarom, roboflow ×2) — all remapped to 2 classes: `with_helmet (0)`, `without_helmet (1)` |
| - **Input**: Top 45% of each rider bounding box (head crop) with 5% lateral padding |
| - **Confidence**: 0.25 |
|
|
| ### `license.pt` — License Plate Localiser |
| - **Type**: YOLO custom, trained on Indian license plates |
| - **Used for**: Detecting the tight bounding box of the license plate within a bike crop |
| - **Confidence**: 0.20 (low threshold to catch partially visible plates) |
|
|
| ### `FSRCNN_x3.pb` — Super-Resolution |
| - **Type**: FSRCNN (Fast Super-Resolution CNN), ×3 scale, TensorFlow/OpenCV DNN |
| - **Used for**: Upscaling small plate crops (often <100px tall) 3× before OCR to improve recognition accuracy |
| |
| ### `paddleocr/` — OCR Engine (PaddleOCR 3.5.0) |
| - **Detection**: `PP-OCRv5_mobile_det` (4.7 MB) — finds text line bounding boxes within the plate crop |
| - **Recognition**: `en_PP-OCRv5_mobile_rec` (7.6 MB) — reads each text line |
| - **Orientation models**: `PP-LCNet_x1_0_doc_ori`, `PP-LCNet_x1_0_textline_ori` — handle rotated plates |
| - **Unwarping**: `UVDoc` — corrects perspective distortion |
| - **API**: Uses the legacy `.ocr()` method (not `.predict()`). Both call the same underlying pipeline, but `.ocr()` uses a compatible inference backend on Windows/Linux CPU without triggering the OneDNN fused_conv2d operator crash present in the newer `.predict()` path |
| - **Post-processing**: Text is uppercased, non-alphanumeric characters stripped, tokens shorter than 2 characters discarded |
| |
| --- |
| |
| ## Offline Operation |
| |
| All model weights are bundled in `./models/`. No internet connection is required at runtime. |
| |
| PaddleOCR 3.5.0 uses [paddlex](https://github.com/PaddlePaddle/PaddleX) internally and looks for models via the `PADDLE_PDX_CACHE_HOME` environment variable. `solution.py` sets this variable to `./models/paddleocr/` **before** any paddle import, so paddlex resolves all models from the bundled path: |
|
|
| ```python |
| os.environ["PADDLE_PDX_CACHE_HOME"] = str(Path(__file__).parent / "models" / "paddleocr") |
| ``` |
|
|
| --- |
|
|
| ## Design Decisions |
|
|
| ### Why two bike detectors? |
| COCO's `motorcycle` class (cls 3) misses many Indian two-wheeler types. The custom `stage1_best.pt` trained on traffic footage recovers these. Boxes from both are merged via NMS. |
|
|
| ### Why depth filtering? |
| In busy street scenes, COCO frequently detects pedestrians on the footpath who share horizontal overlap with a detected bike. Depth-Anything V2 provides a proxy for Z-distance; persons whose median depth differs from the bike's median depth by more than 35% are excluded from association. |
|
|
| ### Why not use PaddleOCR's server detection model? |
| `PP-OCRv5_server_det` is 84.3 MB — bundling it would push the total over 250 MB. Instead, `license.pt` performs the coarse plate localisation (narrowing the search area to ~125×90 px), then `PP-OCRv5_mobile_det` (4.7 MB) finds individual text lines within that small crop, and `en_PP-OCRv5_mobile_rec` reads them. This two-stage localisation gives equivalent quality at a fraction of the size. |
|
|
| ### Why store depth model as fp16? |
| `model.safetensors` converted from fp32 (94.6 MB) to fp16 (47.3 MB) at submission time using `safetensors.torch`. At runtime the model is loaded as fp32 (`dtype=torch.float32`) because x86 CPUs have no native fp16 compute units — running fp16 tensors on CPU causes a 10× slowdown. The disk saving is free; the compute cost is zero. |
|
|
| ### Fallback for missing riders |
| If no COCO person is associated with a detected bike (e.g., very small image, occluded rider), one rider with no helmet is assumed. This is a conservative choice — it risks a false positive but never misses a genuine violation. |
|
|
| --- |
|
|
| ## Constraints Compliance |
|
|
| | Constraint | Status | |
| |---|---| |
| | Model size ≤ 250 MB | ✅ 194.6 MB | |
| | No VLMs > 1B parameters | ✅ Largest model is Depth-Anything V2 Small (~24M params) | |
| | Fully offline execution | ✅ All weights in `./models/`, `PADDLE_PDX_CACHE_HOME` redirected | |
| | `TrafficViolationDetector` interface | ✅ `__init__(model_dir)` + `predict(image_path) → dict` | |
| | Stateless `predict()` | ✅ No mutable shared state between calls | |
| | Error handling | ✅ All exceptions caught; returns `{"violations": []}` on failure | |
|
|
| --- |
|
|
| ## Performance (Local Windows CPU) |
|
|
| | Metric | Value | |
| |---|---| |
| | Init time (cold start) | ~3–4 s | |
| | Inference — simple scene (1–2 bikes) | ~4–5 s | |
| | Inference — dense scene (8+ bikes) | ~10–12 s | |
|
|
| > **Note**: The evaluation server runs Linux with a faster CPU; inference times are expected to be lower. Depth estimation (Depth-Anything V2) is the primary bottleneck on CPU. |
|
|
|
|
