metadata
license: mit
base_model: CIDAS/clipseg-rd64-refined
tags:
- image-segmentation
- semantic-segmentation
- computer-vision
- crack-detection
- infrastructure
- clipseg
datasets:
- roboflow
metrics:
- iou
- dice
CrackSeg
Fine-tuned CLIPSeg for pixel-wise surface crack detection. Given an image of any surface, the model returns a binary segmentation mask highlighting crack regions.
Model Performance
| Metric | Score |
|---|---|
| Dice Score | 0.612 |
| mIoU | 0.716 |
Live Demo
Try it on HuggingFace Spaces.
Training Details
- Dataset: 14,000+ crack images (Roboflow, COCO format)
- Fine-tuning: Partial — decoder fully unfrozen + last 2 layers of CLIP vision encoder + last 1 layer of CLIP text encoder
- Loss: Focal Loss (α=0.75, γ=2.0)
- Optimizer: AdamW with differential learning rates
- Scheduler: CosineAnnealingLR
- Early stopping: patience = 5
Usage
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoProcessor, CLIPSegForImageSegmentation
from PIL import Image
processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined")
model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined")
path = hf_hub_download(repo_id="primus29/crackseg", filename="best_model.pth")
checkpoint = torch.load(path, map_location="cpu", weights_only=False)
model.load_state_dict(checkpoint['model_state_dict'])
model.eval()
image = Image.open("your_image.jpg")
inputs = processor(text="segment crack", images=image, return_tensors="pt", padding=True)
with torch.no_grad():
outputs = model(**inputs)
mask = torch.sigmoid(outputs.logits).squeeze()
mask = (mask > 0.5).float()
Limitations
- Shadow regions can be misidentified as cracks
- Performance degrades on very thin hairline cracks
- Trained primarily on surface/concrete crack data; may not generalize to all materials