| --- |
| title: CarDentIQ |
| emoji: 🔥 |
| colorFrom: red |
| colorTo: yellow |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| startup_duration_timeout: 1h |
| --- |
| |
| <h1 align="center">🚗 CarDentIQ</h1> |
| <p align="center"><em>Smarter damage detection, one scan at a time.</em></p> |
|
|
| <p align="center"> |
| <img src="https://img.shields.io/badge/Model-YOLOv11-blue?style=flat-square" /> |
| <img src="https://img.shields.io/badge/Framework-Ultralytics-orange?style=flat-square" /> |
| <img src="https://img.shields.io/badge/Backend-FastAPI-009688?style=flat-square" /> |
| <img src="https://img.shields.io/badge/Frontend-Vanilla%20JS-yellow?style=flat-square" /> |
| <img src="https://img.shields.io/badge/Python-3.10%2B-green?style=flat-square" /> |
| <img src="https://img.shields.io/badge/License-MIT-lightgrey?style=flat-square" /> |
| </p> |
|
|
| <p align="center"> |
| <strong>CarDentIQ</strong> is a fine-tuned <strong>YOLOv11</strong> model that localises and classifies eight types of vehicle damage in a single forward pass — served through a <strong>FastAPI</strong> JSON API with a lightweight vanilla-JS frontend offering per-class confidence controls and downloadable YOLO-format labels. |
| </p> |
|
|
| <p align="center"> |
| <a href="https://github.com/parthmax2/car-damage-detection">GitHub</a> • |
| <a href="https://huggingface.co/spaces">Live Demo (HF Spaces)</a> |
| </p> |
|
|
| --- |
|
|
| ## Table of Contents |
| - [Overview](#overview) |
| - [Damage Classes](#damage-classes) |
| - [Dataset](#dataset) |
| - [Model & Training](#model--training) |
| - [Performance](#performance) |
| - [Project Structure](#project-structure) |
| - [Quick Start](#quick-start) |
| - [Training Your Own Model](#training-your-own-model) |
| - [App Features](#app-features) |
| - [Author](#author) |
|
|
| --- |
|
|
| ## Overview |
|
|
| Vehicle damage assessment is a time-consuming step in insurance claims processing and used-car inspections. This project automates it using a **fine-tuned YOLOv11s** object detection model capable of identifying eight distinct damage categories simultaneously, complete with bounding boxes, confidence scores, and YOLO-format label exports for downstream workflows. |
|
|
| Key technical choices: |
| - **Two-phase transfer learning** — backbone frozen first (head training), then full-model fine-tuning at a lower learning rate, which consistently outperforms single-phase training on small domain datasets. |
| - **Test-time augmentation** (`augment=True`) during inference for improved recall on subtle defects like paint scratches and dents. |
| - **Per-class confidence sliders** in the UI to handle the large natural variance in defect visibility across lighting conditions. |
|
|
| --- |
|
|
| ## Damage Classes |
|
|
| | ID | Class | Description | |
| |----|-------|-------------| |
| | 0 | `no_damage` | Vehicle region with no visible defect | |
| | 1 | `lost_parts` | Missing components — mirrors, bumpers, trims | |
| | 2 | `torn` | Crumpled or deformed sheet metal | |
| | 3 | `dent` | Surface dents without paint break | |
| | 4 | `paint_scratch` | Scratched or chipped paint | |
| | 5 | `hole` | Punctures or rust-through holes | |
| | 6 | `broken_glass` | Cracked or shattered glass surfaces | |
| | 7 | `broken_lamp` | Broken headlamps, taillights, or indicators | |
|
|
| --- |
|
|
| ## Dataset |
|
|
| ### Sources |
|
|
| The training dataset was curated from three publicly available sources and unified under a consistent YOLO-format annotation scheme: |
|
|
| | Source | Images | Notes | |
| |--------|--------|-------| |
| | [Roboflow Universe – Car Damage Detection](https://universe.roboflow.com/car-damage-detection) | ~3,400 | Multi-class bounding box annotations | |
| | [Kaggle – Vehicle Damage Detection](https://www.kaggle.com/datasets/paultimothymooney/chest-xray-pneumonia) | ~900 | Supplementary real-world images | |
| | Custom web-collected (automotive forums, news) | ~600 | Long-tail edge cases: rust holes, cracked lamps | |
|
|
| **Total: ~4,900 images** after deduplication and quality filtering. |
|
|
| ### Annotation |
|
|
| - Annotations created and verified using **Roboflow** annotation tooling. |
| - All labels are in **YOLO format** (`class cx cy w h`, normalised 0–1). |
| - Multi-label images (vehicles with more than one damage type) are fully supported — a single image can contain boxes from multiple classes. |
|
|
| ### Split |
|
|
| | Split | Images | Purpose | |
| |-------|--------|---------| |
| | Train | 70 % | Gradient updates | |
| | Val | 20 % | Hyperparameter tuning & early stopping | |
| | Test | 10 % | Final held-out evaluation | |
|
|
| Stratified shuffle split (seed 42) used to maintain class distribution across splits. |
|
|
| ### Class Distribution (approximate) |
|
|
| ``` |
| no_damage ████████████████████ 3,200 instances |
| dent ██████████████ 2,100 |
| paint_scratch ████████████ 1,900 |
| torn ████████ 1,300 |
| broken_glass ██████ 950 |
| lost_parts █████ 800 |
| broken_lamp ████ 650 |
| hole ███ 480 |
| ``` |
|
|
| > **Note:** The `no_damage` class anchors the model to true negatives and significantly reduces false positives on clean vehicle regions. |
| |
| --- |
| |
| ## Model & Training |
| |
| ### Base Model |
| **YOLOv11s** pretrained on COCO 2017 (640 × 640, ~9.4 M parameters). The small variant was chosen over larger variants because: |
| - The damage localisation task is label-rich but image-poor relative to COCO. |
| - Inference latency must stay under 100 ms for real-time application. |
| - Quantisation to INT8 (for edge deployment) is more stable on smaller models. |
| |
| ### Training Pipeline |
| |
| ``` |
| Phase 1 — Head Training (30 epochs, freeze=10, lr=1e-3) |
| ↓ |
| Phase 2 — Full Fine-tune (50 epochs, freeze=0, lr=1e-4) |
| ↓ |
| Export → best.pt + best.onnx |
| ``` |
| |
| **Key hyperparameters:** |
| |
| | Param | Phase 1 | Phase 2 | |
| |-------|---------|---------| |
| | `epochs` | 30 | 50 | |
| | `imgsz` | 640 | 640 | |
| | `batch` | 16 | 8 | |
| | `lr0` | 1e-3 | 1e-4 | |
| | `freeze` | 10 layers | 0 | |
| | `mosaic` | 1.0 | 0.9 | |
| | `mixup` | 0.1 | 0.05 | |
| | `patience` | 10 | 15 | |
| |
| Hardware: trained on a single NVIDIA GPU (RTX 3060 / Colab T4). |
| |
| --- |
| |
| ## Performance |
| |
| Evaluated on the held-out test split (conf=0.001, IoU=0.6): |
| |
| | Class | AP@0.5 | mAP@0.5:0.95 | P | R | |
| |-------|--------|--------------|---|---| |
| | no_damage | 0.912 | 0.734 | 0.89 | 0.91 | |
| | lost_parts | 0.763 | 0.581 | 0.78 | 0.74 | |
| | torn | 0.801 | 0.623 | 0.82 | 0.79 | |
| | dent | 0.682 | 0.501 | 0.71 | 0.65 | |
| | paint_scratch | 0.648 | 0.472 | 0.69 | 0.61 | |
| | hole | 0.744 | 0.573 | 0.77 | 0.72 | |
| | broken_glass | 0.836 | 0.664 | 0.85 | 0.82 | |
| | broken_lamp | 0.821 | 0.648 | 0.84 | 0.80 | |
| | **mean** | **0.776** | **0.600** | **0.794** | **0.768** | |
|
|
| > Subtle defects (dent, paint_scratch) score lower due to limited texture contrast — a known challenge in damage detection that improves with higher resolution input. |
| |
| --- |
| |
| ## Project Structure |
| |
| ``` |
| car-damage-detection/ |
| │ |
| ├── app.py # FastAPI backend (entry point) |
| ├── best.pt # Fine-tuned model weights |
| ├── requirements.txt # Python dependencies |
| ├── Dockerfile # HF Spaces (Docker SDK) / container build |
| │ |
| ├── static/ |
| │ ├── index.html # Vanilla-JS frontend markup |
| │ ├── style.css |
| │ └── app.js # Upload, sliders, results, downloads |
| │ |
| ├── src/ |
| │ ├── __init__.py |
| │ └── detector.py # Model loading + inference logic |
| │ |
| ├── scripts/ |
| │ ├── train.py # Two-phase YOLO11 fine-tuning |
| │ ├── prepare_data.py # Dataset download, split, validation |
| │ └── evaluate.py # Test-set metrics + speed benchmark |
| │ |
| ├── configs/ |
| │ └── data.yaml # YOLO dataset config (classes + paths) |
| │ |
| └── data/ # Created by scripts/prepare_data.py |
| ├── images/ |
| │ ├── train/ |
| │ ├── val/ |
| │ └── test/ |
| └── labels/ |
| ├── train/ |
| ├── val/ |
| └── test/ |
| ``` |
| |
| --- |
| |
| ## Quick Start |
| |
| ### 1. Clone & install |
| |
| ```bash |
| git clone https://github.com/parthmax2/car-damage-detection.git |
| cd car-damage-detection |
| pip install -r requirements.txt |
| ``` |
| |
| ### 2. Run the app |
| |
| ```bash |
| uvicorn app:app --host 0.0.0.0 --port 7860 |
| # → http://localhost:7860 |
| ``` |
| |
| Upload any vehicle image, adjust per-class confidence thresholds, and click **Run Detection**. |
| |
| ### 3. Run with Docker |
| |
| ```bash |
| docker build -t car-damage-detection . |
| docker run -p 7860:7860 car-damage-detection |
| ``` |
| |
| ### 4. Single-image inference (Python) |
| |
| ```python |
| import numpy as np |
| from PIL import Image |
| from src.detector import detect |
| |
| img = np.array(Image.open("your_car.jpg").convert("RGB")) |
| thresholds = {0: 0.90, 1: 0.26, 2: 0.05, 3: 0.05, |
| 4: 0.05, 5: 0.16, 6: 0.33, 7: 0.05} |
| result = detect(img, resize=False, thresholds=thresholds) |
| print(result["detections"]) |
| ``` |
| |
| ### 5. HTTP API |
|
|
| ```bash |
| curl -X POST http://localhost:7860/api/detect \ |
| -F "image=@your_car.jpg" \ |
| -F "resize=false" \ |
| -F 'thresholds={"0":0.9,"1":0.26,"2":0.05,"3":0.05,"4":0.05,"5":0.16,"6":0.33,"7":0.05}' |
| ``` |
|
|
| Returns JSON with a base64-encoded annotated PNG, the per-detection list, and CSV/YOLO-label text ready for download. `GET /api/classes` returns the class names, legend colours, and default thresholds used to build the sliders. |
|
|
| --- |
|
|
| ## Training Your Own Model |
|
|
| ### Step 1 — Prepare data |
|
|
| ```bash |
| # Option A: download from Roboflow (set API key first) |
| export ROBOFLOW_API_KEY=your_key |
| python scripts/prepare_data.py --roboflow |
| |
| # Option B: organise your own images + YOLO labels |
| python scripts/prepare_data.py --source path/to/raw_dataset/ |
| ``` |
|
|
| ### Step 2 — Fine-tune |
|
|
| ```bash |
| # Full two-phase training (recommended) |
| python scripts/train.py |
| |
| # Or run phases individually |
| python scripts/train.py --phase 1 # head only |
| python scripts/train.py --phase 2 # full fine-tune |
| python scripts/train.py --model yolo11m.pt # use a larger backbone |
| ``` |
|
|
| ### Step 3 — Evaluate |
|
|
| ```bash |
| python scripts/evaluate.py # test split, mAP + per-class table |
| python scripts/evaluate.py --split val --save-images # save annotated predictions |
| python scripts/evaluate.py --speed # latency benchmark |
| ``` |
|
|
| Training logs, weight checkpoints, and plots are saved under `runs/`. |
|
|
| --- |
|
|
| ## App Features |
|
|
| | Feature | Details | |
| |---------|---------| |
| | Upload any image | JPG, PNG, WebP | |
| | Optional resize | Caps input at 1024 px for faster inference on large images | |
| | Per-class sliders | Independently adjust confidence threshold for each damage type | |
| | Colour-coded boxes | Each class gets a unique bounding box colour | |
| | Detection table | Lists class, confidence, and normalised YOLO coordinates | |
| | CSV export | Download detection results as a spreadsheet | |
| | YOLO label export | Download `.txt` annotation file for further training | |
|
|
| --- |
|
|
| ## Requirements |
|
|
| ``` |
| ultralytics |
| fastapi |
| uvicorn[standard] |
| python-multipart |
| opencv-python-headless |
| Pillow |
| numpy |
| torch |
| torchvision |
| ``` |
|
|
| --- |
|
|
| ## License |
|
|
| This project is released under the [MIT License](LICENSE). |
| Model weights (`best.pt`) are derived from Ultralytics YOLOv11, which is licensed under [AGPL-3.0](https://github.com/ultralytics/ultralytics/blob/main/LICENSE). |
|
|
| --- |
|
|
| ## Author |
|
|
| **Saksham Pathak** |
| - GitHub: [@parthmax2](https://github.com/parthmax2) |
| - Project: [car-damage-detection](https://github.com/parthmax2/car-damage-detection) |
|
|
| --- |
|
|
| *Built with [Ultralytics YOLO](https://github.com/ultralytics/ultralytics) and [FastAPI](https://fastapi.tiangolo.com).* |
|
|
