Add pothole detection as new menu type with separate engine and research plan

Pothole_Detection_Integration_Plan.md

@@ -0,0 +1,92 @@
+# Pothole Detection Integration (Research + Architecture)
+
+## 1) Computer vision approaches for road damage detection
+
+Typical successful families:
+
+- **Object detection (bounding boxes)**: YOLOv5/YOLOv8, Faster R-CNN  
+  - Pros: simple outputs, fast, easy UI.  
+  - Cons: boxes are coarse; struggles with thin cracks and complex shapes.
+
+- **Instance segmentation**: Mask R-CNN, YOLACT  
+  - Pros: tighter region boundary and size estimation.  
+  - Cons: heavier models, more training complexity.
+
+- **Semantic segmentation**: U-Net / DeepLabv3+ / SegFormer  
+  - Pros: best for pixel-level damage maps, severity estimation.  
+  - Cons: needs mask labels; inference cost.
+
+- **Two-stage pipelines**:
+  1) road surface ROI extraction (segment road), then  
+  2) damage detection inside road only  
+  - Pros: reduces false positives (buildings, shadows, non-road textures).
+
+## 2) Datasets and pretrained models (starting points)
+
+Common public datasets (road damage + potholes):
+
+- **RDD (Road Damage Dataset / Road Damage Detection)**  
+  Includes potholes and other damage classes from multiple countries.
+
+- **Pothole-600 / Pothole datasets on Kaggle**  
+  Smaller but useful for prototyping.
+
+- **CrackForest / CFD / other crack datasets**  
+  More focused on cracks; can help pretraining for surface defects.
+
+Practical approach:
+- Use a model pretrained on COCO, then fine-tune on RDD/pothole datasets.
+- For best results, fine-tune on **your region-specific imagery** (road texture and lighting differs).
+
+## 3) Feasibility: drone vs satellite vs vehicle camera
+
+- **Vehicle camera (recommended)**:
+  - Highest feasibility for potholes.
+  - Typical resolution and perspective supports pothole features.
+
+- **Drone (good)**:
+  - Works well at low altitude with good GSD (cm/px).
+  - Requires flight plan and stable capture.
+
+- **Satellite (usually not feasible)**:
+  - Most satellite imagery is too low resolution for potholes.
+  - Only very high-res (sub-10cm) commercial imagery could work, and still hard due to shadows and angle.
+
+## 4) Detection pipeline (integrated with current system)
+
+Implemented integration strategy:
+
+- Add **Pothole Detection** as another detection type in the menu.
+- Route through the existing `POST /api/detect` using:
+  - `detection_type=pothole_detection`
+  - `pothole_model=<selected>`
+- Separate engine module:
+  - `app/pothole_engine.py`
+
+### Starter logic implemented (Rule-Based v1)
+
+For fast CPU MVP (vehicle/drone imagery):
+- shadow/dark region score (local brightness drop)
+- rough texture score (Laplacian roughness)
+- edge score (Canny)
+- fuse + sensitivity percentile threshold
+- region extraction + severity/confidence
+
+## 5) Model architectures to implement next
+
+- **YOLOv8n (boxes)** for fast detection and scalable deployment.
+- **SegFormer-b0 / U-Net** for pixel-level damage mapping.
+- Optional road ROI segmentation first to reduce false positives.
+
+## 6) Immediate next steps for dataset preprocessing / feature extraction
+
+1. Define input standard: camera height, FOV, resolution, and capture protocol.
+2. Build a labeled dataset:
+   - bounding boxes or masks for potholes,
+   - metadata: day/night, wet/dry, shadows.
+3. Add preprocessing:
+   - road ROI extraction,
+   - illumination normalization,
+   - motion blur handling.
+4. Train baseline YOLO model and integrate as `pothole_model=YOLOv8`.
+

README.md

@@ -17,6 +17,7 @@ Standalone web application for satellite image change detection with **user acco
 - **Database** — SQLite (or set `DATABASE_URL` for PostgreSQL); stores users and detection runs
 - **Change detection** — Same model as the original app: AI-based, image difference, feature-based, hybrid
 - **Detection menu** — Choose between General Change Detection and Landslide Detection (Uttarakhand starter)
+- **Pothole detection** — Separate detection type for road damage (starter pipeline + future model hook)
 - **Object classification** — Changed regions labeled as Water, Vegetation/Tree, Building, Road, Bare Ground/Soil
 - **History** — List of past runs with overlay images and stats
 - **UI** — Single-page app with a dark, “control room” style and teal accents
@@ -64,6 +65,11 @@ Standalone web application for satellite image change detection with **user acco
   - Dataset preprocessing starter: `app/landslide_preprocessing.py`
   - Planning/research brief: `Landslide_Detection_Uttarakhand_Integration_Plan.md`
 
+- **Pothole module**:
+  - Integrated at runtime through the same `/api/detect` endpoint using `detection_type=pothole_detection`.
+  - Engine code: `app/pothole_engine.py`
+  - Planning/research brief: `Pothole_Detection_Integration_Plan.md`
+
 ## Project layout
 
 ```

app/main.py

@@ -227,6 +227,7 @@ async def detect(
     method: str = Form("AI-Based Deep Learning"),
     detection_type: str = Form("change_detection"),
     landslide_model: str = Form("Rule-Based v1"),
+    pothole_model: str = Form("Rule-Based v1"),
     title: str = Form("Untitled run"),
     zone: str = Form(""),
     village: str = Form(""),
@@ -278,6 +279,16 @@ async def detect(
             detection_sensitivity=detection_sensitivity,
             min_region_area=min_region_area,
         )
+    elif detection_type == "pothole_detection":
+        from .pothole_engine import run_pothole_detection
+        method = f"Pothole - {pothole_model}"
+        change_mask, result_image, stats, change_regions = run_pothole_detection(
+            before_pil,
+            after_pil,
+            model_name=pothole_model,
+            detection_sensitivity=detection_sensitivity,
+            min_region_area=min_region_area,
+        )
     else:
         detection_type = "change_detection"
         from .detection_engine import run_detection
@@ -576,3 +587,8 @@ def detect_change_page():
 @app.get("/detect/landslide", response_class=HTMLResponse)
 def detect_landslide_page():
     return index()
+
+
+@app.get("/detect/pothole", response_class=HTMLResponse)
+def detect_pothole_page():
+    return index()

app/pothole_engine.py

@@ -0,0 +1,191 @@
+"""
+Pothole / road damage detection starter engine.
+
+Goal: separate pipeline that can evolve to a real model (YOLO/Mask R-CNN/SegFormer).
+This initial version is a CPU-friendly heuristic detector designed for vehicle/drone imagery.
+
+Notes:
+- Satellite imagery is generally too coarse for potholes unless very high resolution (<10 cm/px).
+- Vehicle camera or low-altitude drone is the realistic input for pothole detection.
+"""
+from __future__ import annotations
+
+import cv2
+import numpy as np
+from PIL import Image
+
+
+def _preprocess(image: Image.Image, max_size: int = 1600) -> np.ndarray:
+    arr = np.array(image.convert("RGB"))
+    h, w = arr.shape[:2]
+    if max(h, w) > max_size:
+        s = max_size / max(h, w)
+        arr = cv2.resize(arr, (max(1, int(w * s)), max(1, int(h * s))), interpolation=cv2.INTER_AREA)
+    return arr
+
+
+def _norm01(x: np.ndarray) -> np.ndarray:
+    x = x.astype(np.float32)
+    lo = float(np.min(x))
+    hi = float(np.max(x))
+    if hi - lo < 1e-8:
+        return np.zeros_like(x, dtype=np.float32)
+    return (x - lo) / (hi - lo)
+
+
+def _road_texture_response(gray: np.ndarray) -> np.ndarray:
+    # Potholes often appear as dark regions with sharp boundaries + rough texture.
+    blur = cv2.GaussianBlur(gray, (5, 5), 0)
+    lap = cv2.Laplacian(blur, cv2.CV_32F, ksize=3)
+    rough = cv2.GaussianBlur(np.abs(lap), (7, 7), 0)
+    return _norm01(rough)
+
+
+def _shadow_score(gray: np.ndarray) -> np.ndarray:
+    # Darker-than-local background regions.
+    local = cv2.GaussianBlur(gray, (31, 31), 0)
+    diff = np.clip((local - gray).astype(np.float32), 0, None)
+    return _norm01(diff)
+
+
+def _edge_score(gray: np.ndarray) -> np.ndarray:
+    med = float(np.median(gray))
+    t1 = int(max(0, 0.66 * med))
+    t2 = int(min(255, 1.33 * med))
+    edges = cv2.Canny(gray, t1, t2)
+    edges = cv2.dilate(edges, np.ones((3, 3), np.uint8), iterations=1)
+    return edges.astype(np.float32) / 255.0
+
+
+def _clean(mask: np.ndarray) -> np.ndarray:
+    m = mask.copy()
+    m = cv2.medianBlur(m, 5)
+    k_open = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
+    k_close = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (9, 9))
+    m = cv2.morphologyEx(m, cv2.MORPH_OPEN, k_open)
+    m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, k_close)
+    return m
+
+
+def _extract_regions(mask: np.ndarray, min_area: int = 220):
+    n, labels, stats, cents = cv2.connectedComponentsWithStats(mask, connectivity=8)
+    h, w = mask.shape[:2]
+    img_area = h * w
+    regs = []
+    rid = 0
+    for i in range(1, n):
+        area = int(stats[i, cv2.CC_STAT_AREA])
+        if area < min_area:
+            continue
+        x = int(stats[i, cv2.CC_STAT_LEFT])
+        y = int(stats[i, cv2.CC_STAT_TOP])
+        bw = int(stats[i, cv2.CC_STAT_WIDTH])
+        bh = int(stats[i, cv2.CC_STAT_HEIGHT])
+        if bw * bh > img_area * 0.25:
+            continue
+        ar = max(bw, bh) / max(1, min(bw, bh))
+        if ar > 6.0:
+            continue
+        cx, cy = cents[i]
+        fill = area / max(1, bw * bh)
+        conf = float(np.clip(0.25 + fill * 0.7, 0.25, 0.95))
+        sev = "minor"
+        if area / img_area > 0.01:
+            sev = "major"
+        elif area / img_area > 0.003:
+            sev = "moderate"
+        rid += 1
+        regs.append(
+            {
+                "id": rid,
+                "area": area,
+                "bbox": (x, y, bw, bh),
+                "center": (int(cx), int(cy)),
+                "object_type": "Pothole / Road Damage",
+                "confidence": conf,
+                "severity": sev,
+                "sub_type": "Pothole",
+                "sub_type_confidence": conf,
+                "estimated_stories": None,
+                "estimated_height_m": None,
+                "construction_stage": None,
+            }
+        )
+    regs.sort(key=lambda r: r["area"], reverse=True)
+    return regs[:80]
+
+
+def _visualize(img: np.ndarray, mask: np.ndarray, regions: list[dict]) -> np.ndarray:
+    out = img.copy().astype(np.float32)
+    m = (mask > 127).astype(np.float32)
+    # Orange overlay for road damage
+    layer = np.zeros_like(out)
+    layer[:, :, 0] = 255
+    layer[:, :, 1] = 165
+    alpha = 0.35
+    for c in range(3):
+        out[:, :, c] = out[:, :, c] * (1 - m * alpha) + layer[:, :, c] * (m * alpha)
+    vis = np.clip(out, 0, 255).astype(np.uint8)
+    for r in regions:
+        x, y, w, h = r["bbox"]
+        cv2.rectangle(vis, (x, y), (x + w, y + h), (0, 140, 255), 2)
+        cv2.putText(vis, str(r["id"]), (x + 4, max(14, y - 4)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1, cv2.LINE_AA)
+    return vis
+
+
+def run_pothole_detection(
+    before_pil: Image.Image,
+    after_pil: Image.Image,
+    model_name: str = "Rule-Based v1",
+    detection_sensitivity: float = 0.6,
+    min_region_area: int | None = None,
+):
+    """
+    Current UI uses (before, after) upload. For potholes, we treat the *after* image as
+    the road image and ignore the before image.
+    """
+    img = _preprocess(after_pil)
+    gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
+
+    rough = _road_texture_response(gray)
+    shadow = _shadow_score(gray)
+    edges = _edge_score(gray)
+
+    fused = 0.45 * shadow + 0.35 * rough + 0.20 * edges
+    fused = cv2.GaussianBlur(fused.astype(np.float32), (7, 7), 0)
+
+    sens = float(np.clip(detection_sensitivity, 0.0, 1.0))
+    q = float(np.clip(0.975 - (sens - 0.5) * 0.10, 0.85, 0.985))
+    thr = float(np.quantile(fused, q))
+    mask = (fused >= thr).astype(np.uint8) * 255
+    mask = _clean(mask)
+
+    if min_region_area is None:
+        min_region_area = int(max(150, min(1200, mask.shape[0] * mask.shape[1] * 0.00005)))
+    regions = _extract_regions(mask, min_area=int(min_region_area))
+    result = _visualize(img, mask, regions)
+
+    total = int(mask.shape[0] * mask.shape[1])
+    changed = int(np.sum(mask > 127))
+    stats = {
+        "total_pixels": total,
+        "changed_pixels": changed,
+        "unchanged_pixels": total - changed,
+        "change_percentage": (changed / total * 100.0) if total else 0.0,
+        "image_width": mask.shape[1],
+        "image_height": mask.shape[0],
+        "threshold_debug": {
+            "method": f"Pothole Detection ({model_name})",
+            "threshold_used": int(np.clip(thr * 255.0, 0, 255)),
+            "threshold_percentile_q": q,
+            "sensitivity": sens,
+        },
+        "params": {
+            "detection_sensitivity": sens,
+            "min_region_area": int(min_region_area),
+            "model_name": model_name,
+            "input": "after_only",
+        },
+    }
+    return mask, result, stats, regions
+

static/js/app.js

@@ -15,6 +15,7 @@ function showView(id) {
 function getDetectionTypeFromPath() {
   const p = (window.location.pathname || '').toLowerCase();
   if (p.includes('/detect/landslide')) return 'landslide_detection';
+  if (p.includes('/detect/pothole')) return 'pothole_detection';
   if (p.includes('/detect/change')) return 'change_detection';
   return null;
 }
@@ -27,7 +28,9 @@ function applyDetectionTypeToUI(type) {
 }
 
 function pathForDetectionType(type) {
-  return type === 'landslide_detection' ? '/detect/landslide' : '/detect/change';
+  if (type === 'landslide_detection') return '/detect/landslide';
+  if (type === 'pothole_detection') return '/detect/pothole';
+  return '/detect/change';
 }
 
 function navigateToDetectionType(type, replace = false) {
@@ -48,6 +51,9 @@ document.getElementById('btn-type-change')?.addEventListener('click', () => {
 document.getElementById('btn-type-landslide')?.addEventListener('click', () => {
   navigateToDetectionType('landslide_detection');
 });
+document.getElementById('btn-type-pothole')?.addEventListener('click', () => {
+  navigateToDetectionType('pothole_detection');
+});
 
 function showError(id, msg) {
   const el = document.getElementById(id);
@@ -232,10 +238,11 @@ function setupUploadZone(inputId, nameId, zoneId, previewId) {
 setupUploadZone('file-before', 'name-before', 'zone-before', 'preview-before');
 setupUploadZone('file-after', 'name-after', 'zone-after', 'preview-after');
 
-// ---- Detection menu (General vs Landslide) ----
+// ---- Detection menu (General vs Landslide vs Pothole) ----
 (function initDetectionMenu() {
   const typeSel = document.getElementById('detect-type');
   const landslideGroup = document.getElementById('landslide-model-group');
+  const potholeGroup = document.getElementById('pothole-model-group');
   const methodGroup = document.getElementById('detect-method')?.closest('.form-group');
   const regGroup = document.getElementById('detect-registration')?.closest('.form-group');
   const normGroup = document.getElementById('detect-normalization')?.closest('.form-group');
@@ -243,10 +250,13 @@ setupUploadZone('file-after', 'name-after', 'zone-after', 'preview-after');
 
   function refresh() {
     const isLandslide = typeSel.value === 'landslide_detection';
+    const isPothole = typeSel.value === 'pothole_detection';
     if (landslideGroup) landslideGroup.classList.toggle('hidden', !isLandslide);
-    if (methodGroup) methodGroup.classList.toggle('hidden', isLandslide);
-    if (regGroup) regGroup.classList.toggle('hidden', isLandslide);
-    if (normGroup) normGroup.classList.toggle('hidden', isLandslide);
+    if (potholeGroup) potholeGroup.classList.toggle('hidden', !isPothole);
+    const hideCore = isLandslide || isPothole;
+    if (methodGroup) methodGroup.classList.toggle('hidden', hideCore);
+    if (regGroup) regGroup.classList.toggle('hidden', hideCore);
+    if (normGroup) normGroup.classList.toggle('hidden', hideCore);
   }
 
   typeSel.addEventListener('change', refresh);
@@ -483,6 +493,9 @@ document.getElementById('form-detect')?.addEventListener('submit', async (e) =>
   if (detectionType === 'landslide_detection') {
     form.append('landslide_model', document.getElementById('landslide-model')?.value || 'Rule-Based v1');
   }
+  if (detectionType === 'pothole_detection') {
+    form.append('pothole_model', document.getElementById('pothole-model')?.value || 'Rule-Based v1');
+  }
   form.append('title', document.getElementById('detect-title').value || 'Untitled run');
   form.append('zone', document.getElementById('detect-zone').value || '');
   form.append('village', document.getElementById('detect-village').value || '');

templates/index.html

@@ -144,6 +144,8 @@
           <button type="button" class="btn btn-primary btn-block" id="btn-type-change">General Change Detection</button>
           <div style="height:0.75rem;"></div>
           <button type="button" class="btn btn-secondary btn-block" id="btn-type-landslide">Landslide Detection (Uttarakhand)</button>
+          <div style="height:0.75rem;"></div>
+          <button type="button" class="btn btn-secondary btn-block" id="btn-type-pothole">Pothole Detection</button>
         </div>
       </div>
     </section>
@@ -181,6 +183,7 @@
               <select id="detect-type">
                 <option value="change_detection" selected>General Change Detection</option>
                 <option value="landslide_detection">Landslide Detection (Uttarakhand)</option>
+                <option value="pothole_detection">Pothole Detection</option>
               </select>
             </div>
             <div class="form-group hidden" id="landslide-model-group">
@@ -189,6 +192,12 @@
                 <option value="Rule-Based v1" selected>Rule-Based v1 (Vegetation Loss + Soil Gain)</option>
               </select>
             </div>
+            <div class="form-group hidden" id="pothole-model-group">
+              <label for="pothole-model">Pothole Model</label>
+              <select id="pothole-model">
+                <option value="Rule-Based v1" selected>Rule-Based v1 (Edges + Shadows + Texture)</option>
+              </select>
+            </div>
           </div>
           <div class="location-row">
             <div class="form-group">
@@ -392,6 +401,6 @@
     </div>
   </div>
 
-  <script src="/static/js/app.js?v=30"></script>
+  <script src="/static/js/app.js?v=31"></script>
 </body>
 </html>