iloveass/kompon-image-gate

Model Card for Kapuni Image Gate (Model 0)

This is a lightweight, 3-class image classification model paired with a heuristic quality pre-check. It serves as the preliminary "gate" for the Kapuni pipeline, deciding whether an uploaded image is a valid building surface (and thus worth passing to downstream damage-assessment models) or an invalid upload (e.g., road, random objects).

Model Details

• Model Type: Image Classification (MobileNetV3-Small backbone)
• Framework: PyTorch / torchvision
• Task: 3-class image classification (building_surface, road_or_pavement, other)
• Input: RGB Image (224x224 pixels), preceded by a heuristic check for blur, brightness, and resolution.
• Output: A predicted class and a confidence score, mapping to an action (ACCEPT or REJECT).
• License: BSD 3-Clause (inherited from torchvision pretrained weights).

Intended Use

Primary Use Case: To act as a low-latency pre-filter for building damage assessment applications. It ensures that heavy, specialized object detection models (Model A) only process relevant images of walls, columns, beams, or slabs, thereby saving compute and providing immediate, tailored feedback to users who upload irrelevant images.

Out-of-Scope Use Cases:

• This model does not detect cracks or assess structural damage severity. It only determines if the surface belongs to a building.
• It is not meant for high-precision content moderation or out-of-distribution adversarial filtering.

How to Get Started with the Model

import torch
import torchvision.transforms as transforms
from torchvision import models
import json
from PIL import Image

# 1. Load config
with open("gate_config.json") as f:
    cfg = json.load(f)

# 2. Initialize model architecture
model = models.mobilenet_v3_small(weights=None)
model.classifier[3] = torch.nn.Linear(model.classifier[3].in_features, len(cfg["classes"]))

# 3. Load weights
model.load_state_dict(torch.load("gate_mobilenetv3.pth", map_location="cpu"))
model.eval()

# 4. Prepare image
tf = transforms.Compose([
    transforms.Resize((cfg["img_size"], cfg["img_size"])),
    transforms.ToTensor(),
    transforms.Normalize(cfg["mean"], cfg["std"]),
])

img = Image.open("your_photo.jpg").convert("RGB")
input_tensor = tf(img).unsqueeze(0)

# 5. Predict
with torch.no_grad():
    logits = model(input_tensor)
    probs = torch.softmax(logits, 1)[0]
    
pred_idx = int(probs.argmax())
confidence = float(probs[pred_idx])
predicted_class = cfg["classes"][pred_idx]

print(f"Class: {predicted_class}, Confidence: {confidence:.2f}")

Training Data

The model was fine-tuned on a balanced dataset of approximately 2,500 images per class, sourced from existing public datasets:

• building_surface: Sourced from HRCDS. Currently composed primarily of damaged concrete walls and building elements.
• road_or_pavement: Sourced from RDD2022 (specifically the India/dashcam subset) to serve as a strong hard-negative for crack-like patterns on roads.
• other: Sourced from COCO (val2017) to capture a wide variety of random objects, scenes, people, and vehicles.

Evaluation and Decision Logic

Because wrongly rejecting a real building crack is a critical failure for a safety tool, the model implements an "Accept-on-Uncertainty" rule.

• If the model's confidence is below TAU (default 0.55), the image is automatically ACCEPTED and passed to the downstream model.
• If the model is highly confident that the image is a road or random object, it is REJECTED.

Limitations and Bias

• Concrete Bias: The building_surface class was trained heavily on damaged concrete (due to the HRCDS source). Clean walls, masonry, or brick might result in lower confidence scores. The Accept-on-Uncertainty rule mitigates this by passing low-confidence images through, but the bias exists natively in the weights.
• Heuristic Quality Check: The pipeline relies on a variance-of-Laplacian check for blur and mean luma for brightness. These thresholds are manual heuristics and may falsely reject stylized or unusually lit images.