| --- |
| license: apache-2.0 |
| tags: |
| - segmentation |
| - prompted-segmentation |
| - clipseg |
| - drywall |
| --- |
| # drywall-clipseg |
|
|
| CLIPSeg fine-tune (HF `CIDAS/clipseg-rd64-refined` backbone, decoder + FiLM |
| unfrozen) for prompted binary segmentation on drywall imagery. Single |
| checkpoint covers two classes selected by text prompt: |
|
|
| - `"segment crack"` → wall-crack mask |
| - `"segment taping area"` → drywall taping-seam mask |
|
|
| ## Test metrics (focal_dice loss, threshold 0.6) |
| |
| | Task | Dice | mIoU | Precision | Recall | |
| | --- | --- | --- | --- | --- | |
| | Crack | 0.672 | 0.531 | — | — | |
| | Taping | 0.727 | 0.587 | — | — | |
| |
| ## Load + predict |
| |
| ```python |
| from huggingface_hub import hf_hub_download |
| import torch |
| from src.models.clipseg_wrapper import CLIPSegFT |
| |
| ckpt_path = hf_hub_download(repo_id="ravindrakapse/drywall-clipseg", filename="best.pt") |
| model = CLIPSegFT(pretrained="CIDAS/clipseg-rd64-refined").cuda() |
| state = torch.load(ckpt_path, map_location="cuda") |
| model.load_state_dict(state["model"]) |
| model.eval() |
| ``` |
| |
| See [`load_models.py`](https://github.com/Ravindrakapse/prompt_segmentation/blob/main/load_models.py) for the full inference pipeline (letterbox + TTA + un-letterbox). |
| |
