Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- segmentation
|
| 5 |
+
- grounding-dino
|
| 6 |
+
- sam
|
| 7 |
+
- drywall
|
| 8 |
+
---
|
| 9 |
+
# drywall-grounded-sam
|
| 10 |
+
|
| 11 |
+
Two-stage prompted segmentation: Grounding DINO (`grounding-dino-tiny`) emits
|
| 12 |
+
bboxes from a text prompt → SAM (`sam-vit-base`) converts boxes → masks → union.
|
| 13 |
+
Both stages fine-tuned on the unified drywall pool (3 epochs each).
|
| 14 |
+
|
| 15 |
+
## Test metrics
|
| 16 |
+
|
| 17 |
+
| Task | Dice | mIoU | Precision | Recall |
|
| 18 |
+
| --- | --- | --- | --- | --- |
|
| 19 |
+
| Crack | 0.601 | 0.463 | 0.637 | 0.725 |
|
| 20 |
+
| Taping | 0.515 | 0.393 | 0.491 | 0.703 |
|
| 21 |
+
|
| 22 |
+
## Load + predict
|
| 23 |
+
|
| 24 |
+
```python
|
| 25 |
+
from huggingface_hub import snapshot_download
|
| 26 |
+
local = snapshot_download(repo_id="ravindrakapse/drywall-grounded-sam")
|
| 27 |
+
# local/gd_finetuned/ -> AutoProcessor + AutoModelForZeroShotObjectDetection
|
| 28 |
+
# local/sam_finetuned/ -> SamProcessor + SamModel
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
See [`load_models.py`](https://github.com/Ravindrakapse/prompt_segmentation/blob/main/load_models.py) for the full pipeline.
|