Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: CIDAS/clipseg-rd64-refined
|
| 4 |
+
tags:
|
| 5 |
+
- image-segmentation
|
| 6 |
+
- semantic-segmentation
|
| 7 |
+
- computer-vision
|
| 8 |
+
- crack-detection
|
| 9 |
+
- infrastructure
|
| 10 |
+
- clipseg
|
| 11 |
+
datasets:
|
| 12 |
+
- roboflow
|
| 13 |
+
metrics:
|
| 14 |
+
- iou
|
| 15 |
+
- dice
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# CrackSeg
|
| 19 |
+
|
| 20 |
+
Fine-tuned [CLIPSeg](https://huggingface.co/CIDAS/clipseg-rd64-refined) for pixel-wise surface crack detection. Given an image of any surface, the model returns a binary segmentation mask highlighting crack regions.
|
| 21 |
+
|
| 22 |
+
## Model Performance
|
| 23 |
+
|
| 24 |
+
| Metric | Score |
|
| 25 |
+
|--------|-------|
|
| 26 |
+
| Dice Score | 0.612 |
|
| 27 |
+
| mIoU | 0.716 |
|
| 28 |
+
|
| 29 |
+
## Live Demo
|
| 30 |
+
|
| 31 |
+
Try it on [HuggingFace Spaces](https://huggingface.co/spaces/primus29/crackseg).
|
| 32 |
+
|
| 33 |
+
## Training Details
|
| 34 |
+
|
| 35 |
+
- **Dataset:** 14,000+ crack images (Roboflow, COCO format)
|
| 36 |
+
- **Fine-tuning:** Partial — decoder fully unfrozen + last 2 layers of CLIP vision encoder + last 1 layer of CLIP text encoder
|
| 37 |
+
- **Loss:** Focal Loss (α=0.75, γ=2.0)
|
| 38 |
+
- **Optimizer:** AdamW with differential learning rates
|
| 39 |
+
- **Scheduler:** CosineAnnealingLR
|
| 40 |
+
- **Early stopping:** patience = 5
|
| 41 |
+
|
| 42 |
+
## Usage
|
| 43 |
+
|
| 44 |
+
```python
|
| 45 |
+
import torch
|
| 46 |
+
from huggingface_hub import hf_hub_download
|
| 47 |
+
from transformers import AutoProcessor, CLIPSegForImageSegmentation
|
| 48 |
+
from PIL import Image
|
| 49 |
+
|
| 50 |
+
processor = AutoProcessor.from_pretrained("CIDAS/clipseg-rd64-refined")
|
| 51 |
+
model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined")
|
| 52 |
+
|
| 53 |
+
path = hf_hub_download(repo_id="primus29/crackseg", filename="best_model.pth")
|
| 54 |
+
checkpoint = torch.load(path, map_location="cpu", weights_only=False)
|
| 55 |
+
model.load_state_dict(checkpoint['model_state_dict'])
|
| 56 |
+
model.eval()
|
| 57 |
+
|
| 58 |
+
image = Image.open("your_image.jpg")
|
| 59 |
+
inputs = processor(text="segment crack", images=image, return_tensors="pt", padding=True)
|
| 60 |
+
|
| 61 |
+
with torch.no_grad():
|
| 62 |
+
outputs = model(**inputs)
|
| 63 |
+
|
| 64 |
+
mask = torch.sigmoid(outputs.logits).squeeze()
|
| 65 |
+
mask = (mask > 0.5).float()
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
## Limitations
|
| 69 |
+
|
| 70 |
+
- Shadow regions can be misidentified as cracks
|
| 71 |
+
- Performance degrades on very thin hairline cracks
|
| 72 |
+
- Trained primarily on surface/concrete crack data; may not generalize to all materials
|