Spaces:
Running
Running
Upload folder using huggingface_hub
Browse files- AGENTS.md +49 -3
- CLAUDE.md +49 -3
- measure_finger.py +0 -12
- script/experiment_sam_mask_quality.py +341 -0
- src/debug_observer.py +0 -4
- src/sam_hand_segmentation.py +16 -16
AGENTS.md
CHANGED
|
@@ -20,8 +20,9 @@ For tasks of **bug fixing**:
|
|
| 20 |
For tasks of **reboot** from a new codex session:
|
| 21 |
1. Read doc/v0/PRD.md, doc/v0/Plan.md, doc/v0/Progress.md for baseline implementation
|
| 22 |
2. Read doc/v1/PRD.md, doc/v1/Plan.md, doc/v1/Progress.md for edge refinement (v1)
|
| 23 |
-
3.
|
| 24 |
-
4.
|
|
|
|
| 25 |
|
| 26 |
## Project Overview
|
| 27 |
|
|
@@ -233,10 +234,12 @@ v1 improves measurement accuracy by replacing contour-based edge detection with
|
|
| 233 |
| Flag | Values | Default | Description |
|
| 234 |
|------|--------|---------|-------------|
|
| 235 |
| `--finger-index` | auto, index, middle, ring, pinky | **index** | Which finger to measure and use for orientation |
|
| 236 |
-
| `--edge-method` | auto, contour, sobel, compare |
|
| 237 |
| `--sobel-threshold` | float | 15.0 | Minimum gradient magnitude |
|
| 238 |
| `--sobel-kernel-size` | 3, 5, 7 | 3 | Sobel kernel size |
|
| 239 |
| `--no-subpixel` | flag | False | Disable sub-pixel refinement |
|
|
|
|
|
|
|
| 240 |
|
| 241 |
### v1 Auto Mode Behavior
|
| 242 |
|
|
@@ -269,6 +272,49 @@ When `--debug` flag used, generates:
|
|
| 269 |
|
| 270 |
---
|
| 271 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 272 |
## Important Technical Details
|
| 273 |
|
| 274 |
### What This Measures
|
|
|
|
| 20 |
For tasks of **reboot** from a new codex session:
|
| 21 |
1. Read doc/v0/PRD.md, doc/v0/Plan.md, doc/v0/Progress.md for baseline implementation
|
| 22 |
2. Read doc/v1/PRD.md, doc/v1/Plan.md, doc/v1/Progress.md for edge refinement (v1)
|
| 23 |
+
3. Read doc/v4/PRD.md, doc/v4/Plan.md, doc/v4/Progress.md for SAM 2.1 integration (card + hand)
|
| 24 |
+
4. Assume this is a continuation of an existing project.
|
| 25 |
+
5. Summarize your understanding of the current state and propose the next concrete step without writing code yet.
|
| 26 |
|
| 27 |
## Project Overview
|
| 28 |
|
|
|
|
| 234 |
| Flag | Values | Default | Description |
|
| 235 |
|------|--------|---------|-------------|
|
| 236 |
| `--finger-index` | auto, index, middle, ring, pinky | **index** | Which finger to measure and use for orientation |
|
| 237 |
+
| `--edge-method` | auto, contour, sobel, mask, compare | **mask** | Edge detection method (see v4 below) |
|
| 238 |
| `--sobel-threshold` | float | 15.0 | Minimum gradient magnitude |
|
| 239 |
| `--sobel-kernel-size` | 3, 5, 7 | 3 | Sobel kernel size |
|
| 240 |
| `--no-subpixel` | flag | False | Disable sub-pixel refinement |
|
| 241 |
+
| `--card-method` | classic, sam | classic | Card detection backend (v4). CLI default is `classic` to avoid surprise SAM weight downloads; web demo forces `sam`. |
|
| 242 |
+
| `--hand-mask` | synthetic, sam | **sam** | Hand mask source (v4). `synthetic` is the MediaPipe landmark convex hull. |
|
| 243 |
|
| 244 |
### v1 Auto Mode Behavior
|
| 245 |
|
|
|
|
| 272 |
|
| 273 |
---
|
| 274 |
|
| 275 |
+
## v4 Architecture (SAM 2.1 Segmentation)
|
| 276 |
+
|
| 277 |
+
v4 replaces the two fragile detection stages in v0/v1 with Meta's Segment Anything 2.1 (Hiera Small, Apache 2.0, ~150 MB). Both SAM calls are prompt-based so CPU inference stays under ~2 s total per image.
|
| 278 |
+
|
| 279 |
+
### What's new in v4
|
| 280 |
+
|
| 281 |
+
- **SAM-based card detection** — `src/sam_card_detection.py::detect_credit_card_sam_prompt()`. Seeds sampled outside the hand mask; each seed fires a positive prompt + negative prompts at every other seed; candidate masks are filtered by rectangularity (≥0.90), aspect ratio (1.586 ± 15%), and area bounds. ~14× faster than the original AMG grid path (which has been removed).
|
| 282 |
+
- **SAM-based hand mask** — `src/sam_hand_segmentation.py::segment_hand_sam()`. Single positive prompt at the palm center (mean of MediaPipe landmarks 0, 5, 9, 13, 17). Replaces the old MediaPipe convex-hull polygon, which was a loose bounding shape — not a mask. Kept available under `hand_data["mask_synthetic"]` for debug.
|
| 283 |
+
- **`mask` edge method** — new default. Measures finger width directly from the SAM mask boundary with no Sobel search. `sobel` is retained as a pure-Sobel path; when explicitly selected it now anchors its per-row search on the SAM boundary (±N pixels) instead of scanning the entire ROI.
|
| 284 |
+
- **Shared SAM backend** (`src/sam_backend.py`) — single `Sam2Model` + `Sam2Processor` singleton shared by card + hand. Tries the local HF cache first (`local_files_only=True`) to avoid HEAD-request retry storms.
|
| 285 |
+
- **Pipeline ordering** — hand mask runs first, its complement is used to seed card detection. This is cheap because SAM hand segmentation is ~0.5 s.
|
| 286 |
+
|
| 287 |
+
### v4 module additions
|
| 288 |
+
|
| 289 |
+
| Module | Purpose |
|
| 290 |
+
|--------|---------|
|
| 291 |
+
| `src/sam_backend.py` | Shared Sam2Model/Sam2Processor singleton |
|
| 292 |
+
| `src/sam_card_detection.py` | Prompt-based SAM card detection + seed helper |
|
| 293 |
+
| `src/sam_hand_segmentation.py` | Prompt-based SAM hand segmentation |
|
| 294 |
+
|
| 295 |
+
### v4 debug additions
|
| 296 |
+
|
| 297 |
+
- SAM card mask and SAM hand mask are blended onto the final debug PNG by `src/visualization.py` so the user can see what was actually measured.
|
| 298 |
+
- `script/validate_sam_card.py` and `script/compare_hand_sam.py` are offline validation/comparison harnesses for the two SAM stages.
|
| 299 |
+
|
| 300 |
+
### v4 defaults
|
| 301 |
+
|
| 302 |
+
| Component | CLI default | Web demo |
|
| 303 |
+
|---|---|---|
|
| 304 |
+
| `--card-method` | `classic` (avoids surprise 150 MB download) | `sam` (hard-coded) |
|
| 305 |
+
| `--hand-mask` | `sam` | `sam` (hard-coded) |
|
| 306 |
+
| `--edge-method` | `mask` | `mask` (hard-coded) |
|
| 307 |
+
|
| 308 |
+
### Ring / pinky handling
|
| 309 |
+
|
| 310 |
+
For outer fingers the ROI is shrunk and rotation is centered on the proximal phalanx rather than the finger midpoint. `mask_only` measurements (i.e., the `mask` edge method) drop invalid rows and hard-fail if too few valid rows remain, rather than silently returning a low-confidence number.
|
| 311 |
+
|
| 312 |
+
### Environment flags
|
| 313 |
+
|
| 314 |
+
- `RING_DISABLE_SUPABASE=1` — opt out of Supabase persistence for local dev runs (the web demo otherwise persists each measurement off the request thread).
|
| 315 |
+
|
| 316 |
+
---
|
| 317 |
+
|
| 318 |
## Important Technical Details
|
| 319 |
|
| 320 |
### What This Measures
|
CLAUDE.md
CHANGED
|
@@ -20,8 +20,9 @@ For tasks of **bug fixing**:
|
|
| 20 |
For tasks of **reboot** from a new codex session:
|
| 21 |
1. Read doc/v0/PRD.md, doc/v0/Plan.md, doc/v0/Progress.md for baseline implementation
|
| 22 |
2. Read doc/v1/PRD.md, doc/v1/Plan.md, doc/v1/Progress.md for edge refinement (v1)
|
| 23 |
-
3.
|
| 24 |
-
4.
|
|
|
|
| 25 |
|
| 26 |
## Project Overview
|
| 27 |
|
|
@@ -233,10 +234,12 @@ v1 improves measurement accuracy by replacing contour-based edge detection with
|
|
| 233 |
| Flag | Values | Default | Description |
|
| 234 |
|------|--------|---------|-------------|
|
| 235 |
| `--finger-index` | auto, index, middle, ring, pinky | **index** | Which finger to measure and use for orientation |
|
| 236 |
-
| `--edge-method` | auto, contour, sobel, compare |
|
| 237 |
| `--sobel-threshold` | float | 15.0 | Minimum gradient magnitude |
|
| 238 |
| `--sobel-kernel-size` | 3, 5, 7 | 3 | Sobel kernel size |
|
| 239 |
| `--no-subpixel` | flag | False | Disable sub-pixel refinement |
|
|
|
|
|
|
|
| 240 |
|
| 241 |
### v1 Auto Mode Behavior
|
| 242 |
|
|
@@ -269,6 +272,49 @@ When `--debug` flag used, generates:
|
|
| 269 |
|
| 270 |
---
|
| 271 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 272 |
## Important Technical Details
|
| 273 |
|
| 274 |
### What This Measures
|
|
|
|
| 20 |
For tasks of **reboot** from a new codex session:
|
| 21 |
1. Read doc/v0/PRD.md, doc/v0/Plan.md, doc/v0/Progress.md for baseline implementation
|
| 22 |
2. Read doc/v1/PRD.md, doc/v1/Plan.md, doc/v1/Progress.md for edge refinement (v1)
|
| 23 |
+
3. Read doc/v4/PRD.md, doc/v4/Plan.md, doc/v4/Progress.md for SAM 2.1 integration (card + hand)
|
| 24 |
+
4. Assume this is a continuation of an existing project.
|
| 25 |
+
5. Summarize your understanding of the current state and propose the next concrete step without writing code yet.
|
| 26 |
|
| 27 |
## Project Overview
|
| 28 |
|
|
|
|
| 234 |
| Flag | Values | Default | Description |
|
| 235 |
|------|--------|---------|-------------|
|
| 236 |
| `--finger-index` | auto, index, middle, ring, pinky | **index** | Which finger to measure and use for orientation |
|
| 237 |
+
| `--edge-method` | auto, contour, sobel, mask, compare | **mask** | Edge detection method (see v4 below) |
|
| 238 |
| `--sobel-threshold` | float | 15.0 | Minimum gradient magnitude |
|
| 239 |
| `--sobel-kernel-size` | 3, 5, 7 | 3 | Sobel kernel size |
|
| 240 |
| `--no-subpixel` | flag | False | Disable sub-pixel refinement |
|
| 241 |
+
| `--card-method` | classic, sam | classic | Card detection backend (v4). CLI default is `classic` to avoid surprise SAM weight downloads; web demo forces `sam`. |
|
| 242 |
+
| `--hand-mask` | synthetic, sam | **sam** | Hand mask source (v4). `synthetic` is the MediaPipe landmark convex hull. |
|
| 243 |
|
| 244 |
### v1 Auto Mode Behavior
|
| 245 |
|
|
|
|
| 272 |
|
| 273 |
---
|
| 274 |
|
| 275 |
+
## v4 Architecture (SAM 2.1 Segmentation)
|
| 276 |
+
|
| 277 |
+
v4 replaces the two fragile detection stages in v0/v1 with Meta's Segment Anything 2.1 (Hiera Small, Apache 2.0, ~150 MB). Both SAM calls are prompt-based so CPU inference stays under ~2 s total per image.
|
| 278 |
+
|
| 279 |
+
### What's new in v4
|
| 280 |
+
|
| 281 |
+
- **SAM-based card detection** — `src/sam_card_detection.py::detect_credit_card_sam_prompt()`. Seeds sampled outside the hand mask; each seed fires a positive prompt + negative prompts at every other seed; candidate masks are filtered by rectangularity (≥0.90), aspect ratio (1.586 ± 15%), and area bounds. ~14× faster than the original AMG grid path (which has been removed).
|
| 282 |
+
- **SAM-based hand mask** — `src/sam_hand_segmentation.py::segment_hand_sam()`. Single positive prompt at the palm center (mean of MediaPipe landmarks 0, 5, 9, 13, 17). Replaces the old MediaPipe convex-hull polygon, which was a loose bounding shape — not a mask. Kept available under `hand_data["mask_synthetic"]` for debug.
|
| 283 |
+
- **`mask` edge method** — new default. Measures finger width directly from the SAM mask boundary with no Sobel search. `sobel` is retained as a pure-Sobel path; when explicitly selected it now anchors its per-row search on the SAM boundary (±N pixels) instead of scanning the entire ROI.
|
| 284 |
+
- **Shared SAM backend** (`src/sam_backend.py`) — single `Sam2Model` + `Sam2Processor` singleton shared by card + hand. Tries the local HF cache first (`local_files_only=True`) to avoid HEAD-request retry storms.
|
| 285 |
+
- **Pipeline ordering** — hand mask runs first, its complement is used to seed card detection. This is cheap because SAM hand segmentation is ~0.5 s.
|
| 286 |
+
|
| 287 |
+
### v4 module additions
|
| 288 |
+
|
| 289 |
+
| Module | Purpose |
|
| 290 |
+
|--------|---------|
|
| 291 |
+
| `src/sam_backend.py` | Shared Sam2Model/Sam2Processor singleton |
|
| 292 |
+
| `src/sam_card_detection.py` | Prompt-based SAM card detection + seed helper |
|
| 293 |
+
| `src/sam_hand_segmentation.py` | Prompt-based SAM hand segmentation |
|
| 294 |
+
|
| 295 |
+
### v4 debug additions
|
| 296 |
+
|
| 297 |
+
- SAM card mask and SAM hand mask are blended onto the final debug PNG by `src/visualization.py` so the user can see what was actually measured.
|
| 298 |
+
- `script/validate_sam_card.py` and `script/compare_hand_sam.py` are offline validation/comparison harnesses for the two SAM stages.
|
| 299 |
+
|
| 300 |
+
### v4 defaults
|
| 301 |
+
|
| 302 |
+
| Component | CLI default | Web demo |
|
| 303 |
+
|---|---|---|
|
| 304 |
+
| `--card-method` | `classic` (avoids surprise 150 MB download) | `sam` (hard-coded) |
|
| 305 |
+
| `--hand-mask` | `sam` | `sam` (hard-coded) |
|
| 306 |
+
| `--edge-method` | `mask` | `mask` (hard-coded) |
|
| 307 |
+
|
| 308 |
+
### Ring / pinky handling
|
| 309 |
+
|
| 310 |
+
For outer fingers the ROI is shrunk and rotation is centered on the proximal phalanx rather than the finger midpoint. `mask_only` measurements (i.e., the `mask` edge method) drop invalid rows and hard-fail if too few valid rows remain, rather than silently returning a low-confidence number.
|
| 311 |
+
|
| 312 |
+
### Environment flags
|
| 313 |
+
|
| 314 |
+
- `RING_DISABLE_SUPABASE=1` — opt out of Supabase persistence for local dev runs (the web demo otherwise persists each measurement off the request thread).
|
| 315 |
+
|
| 316 |
+
---
|
| 317 |
+
|
| 318 |
## Important Technical Details
|
| 319 |
|
| 320 |
### What This Measures
|
measure_finger.py
CHANGED
|
@@ -1523,18 +1523,6 @@ def _draw_multi_finger_debug(
|
|
| 1523 |
valid_rows = edge_data["valid_rows"]
|
| 1524 |
x_min, y_min, x_max, y_max = roi_bounds
|
| 1525 |
|
| 1526 |
-
# ROI box (transform corners to canonical)
|
| 1527 |
-
roi_corners = np.array([
|
| 1528 |
-
[x_min, y_min], [x_max, y_min],
|
| 1529 |
-
[x_max, y_max], [x_min, y_max],
|
| 1530 |
-
], dtype=np.float64)
|
| 1531 |
-
if inv_mat is not None:
|
| 1532 |
-
roi_hom = np.hstack([roi_corners, np.ones((4, 1))]).T
|
| 1533 |
-
roi_can = (inv_mat @ roi_hom).T.astype(np.int32).reshape((-1, 1, 2))
|
| 1534 |
-
else:
|
| 1535 |
-
roi_can = roi_corners.astype(np.int32).reshape((-1, 1, 2))
|
| 1536 |
-
cv2.polylines(vis, [roi_can], isClosed=True, color=color, thickness=1, lineType=cv2.LINE_AA)
|
| 1537 |
-
|
| 1538 |
# Edge dots and width measurement lines
|
| 1539 |
valid_count = int(np.sum(valid_rows))
|
| 1540 |
line_spacing = max(1, valid_count // 20)
|
|
|
|
| 1523 |
valid_rows = edge_data["valid_rows"]
|
| 1524 |
x_min, y_min, x_max, y_max = roi_bounds
|
| 1525 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1526 |
# Edge dots and width measurement lines
|
| 1527 |
valid_count = int(np.sum(valid_rows))
|
| 1528 |
line_spacing = max(1, valid_count // 20)
|
script/experiment_sam_mask_quality.py
ADDED
|
@@ -0,0 +1,341 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Experiment: isolate the cause of the 'staircase' edges on our SAM 2.1 hand mask.
|
| 3 |
+
|
| 4 |
+
Runs five configurations of SAM 2.1 Hiera Small against a single sample image,
|
| 5 |
+
seeded by the MediaPipe palm center. All five variants feed the *same* pixel
|
| 6 |
+
prompts to the *same* model; only the inference input resolution and the
|
| 7 |
+
mask-upscale strategy differ.
|
| 8 |
+
|
| 9 |
+
Configurations:
|
| 10 |
+
|
| 11 |
+
A baseline 1024-long-side hard-mask + INTER_NEAREST upscale (current)
|
| 12 |
+
B lin_hard 1024-long-side hard-mask + INTER_LINEAR + rethresh
|
| 13 |
+
C soft 1024-long-side raw logits -> bilinear -> threshold full-res
|
| 14 |
+
D native_nn native hard-mask + INTER_NEAREST (no upscale needed)
|
| 15 |
+
E native_sft native raw logits -> bilinear -> threshold
|
| 16 |
+
|
| 17 |
+
For each variant the script saves:
|
| 18 |
+
- full-resolution binary mask PNG
|
| 19 |
+
- hand overlay with yellow mask + green palm prompt dot
|
| 20 |
+
- 600x600 fingertip crop centered on the middle-finger tip (landmark 12)
|
| 21 |
+
so the staircase vs smooth comparison is visible at a glance
|
| 22 |
+
|
| 23 |
+
It also prints a small table:
|
| 24 |
+
- perimeter (px) cv2.arcLength of the largest contour
|
| 25 |
+
- iso ratio perimeter / sqrt(area) -- higher = more jagged
|
| 26 |
+
- rel. to baseline (%) iso ratio relative to config A
|
| 27 |
+
|
| 28 |
+
Usage:
|
| 29 |
+
python script/experiment_sam_mask_quality.py \\
|
| 30 |
+
--input input/sample-04-12/card_2.jpg \\
|
| 31 |
+
--output-dir output/sam_mask_quality
|
| 32 |
+
"""
|
| 33 |
+
from __future__ import annotations
|
| 34 |
+
|
| 35 |
+
import argparse
|
| 36 |
+
import sys
|
| 37 |
+
import time
|
| 38 |
+
from pathlib import Path
|
| 39 |
+
from typing import Dict, List, Optional, Tuple
|
| 40 |
+
|
| 41 |
+
import cv2
|
| 42 |
+
import numpy as np
|
| 43 |
+
|
| 44 |
+
# Add repo root so we can import src.*
|
| 45 |
+
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
| 46 |
+
|
| 47 |
+
from src.finger_segmentation import segment_hand
|
| 48 |
+
from src.sam_backend import get_sam2
|
| 49 |
+
from src.sam_hand_segmentation import palm_center_from_landmarks
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
# ----------------------------------------------------------------------------
|
| 53 |
+
# SAM inference variants
|
| 54 |
+
# ----------------------------------------------------------------------------
|
| 55 |
+
|
| 56 |
+
def _run_sam(
|
| 57 |
+
image_bgr: np.ndarray,
|
| 58 |
+
palm_xy: Tuple[int, int],
|
| 59 |
+
inference_long_side: Optional[int],
|
| 60 |
+
upscale_mode: str,
|
| 61 |
+
) -> Tuple[np.ndarray, float, float]:
|
| 62 |
+
"""Run SAM 2.1 with controlled inference resolution and upscale path.
|
| 63 |
+
|
| 64 |
+
Args:
|
| 65 |
+
image_bgr: Full-resolution BGR image (canonical orientation).
|
| 66 |
+
palm_xy: (x, y) pixel coords of palm center in the full-res image.
|
| 67 |
+
inference_long_side: If set, downscale so long-side equals this value.
|
| 68 |
+
If None, feed native resolution.
|
| 69 |
+
upscale_mode: One of:
|
| 70 |
+
- "nearest_hard": post_process_masks -> INTER_NEAREST to full res.
|
| 71 |
+
- "linear_hard": post_process_masks -> INTER_LINEAR -> re-threshold.
|
| 72 |
+
- "soft": raw pred_masks (256x256) -> bilinear to full res
|
| 73 |
+
-> threshold at 0.0.
|
| 74 |
+
|
| 75 |
+
Returns:
|
| 76 |
+
(mask_full: bool HxW, iou_score: float, infer_seconds: float)
|
| 77 |
+
"""
|
| 78 |
+
import torch
|
| 79 |
+
import torch.nn.functional as F
|
| 80 |
+
from PIL import Image as PILImage
|
| 81 |
+
|
| 82 |
+
h_full, w_full = image_bgr.shape[:2]
|
| 83 |
+
long_side = max(h_full, w_full)
|
| 84 |
+
|
| 85 |
+
if inference_long_side is None or long_side <= inference_long_side:
|
| 86 |
+
scaled_bgr = image_bgr
|
| 87 |
+
scale_back = 1.0
|
| 88 |
+
else:
|
| 89 |
+
s = inference_long_side / long_side
|
| 90 |
+
new_w = int(round(w_full * s))
|
| 91 |
+
new_h = int(round(h_full * s))
|
| 92 |
+
scaled_bgr = cv2.resize(image_bgr, (new_w, new_h), interpolation=cv2.INTER_AREA)
|
| 93 |
+
scale_back = 1.0 / s
|
| 94 |
+
|
| 95 |
+
scaled_rgb = cv2.cvtColor(scaled_bgr, cv2.COLOR_BGR2RGB)
|
| 96 |
+
pil = PILImage.fromarray(scaled_rgb)
|
| 97 |
+
|
| 98 |
+
scale_down = 1.0 / scale_back
|
| 99 |
+
palm_scaled = [
|
| 100 |
+
int(round(palm_xy[0] * scale_down)),
|
| 101 |
+
int(round(palm_xy[1] * scale_down)),
|
| 102 |
+
]
|
| 103 |
+
|
| 104 |
+
model, processor = get_sam2()
|
| 105 |
+
|
| 106 |
+
t0 = time.time()
|
| 107 |
+
inputs = processor(
|
| 108 |
+
images=pil,
|
| 109 |
+
input_points=[[[palm_scaled]]],
|
| 110 |
+
input_labels=[[[1]]],
|
| 111 |
+
return_tensors="pt",
|
| 112 |
+
)
|
| 113 |
+
with torch.inference_mode():
|
| 114 |
+
outputs = model(**inputs, multimask_output=True)
|
| 115 |
+
|
| 116 |
+
pred_masks = outputs.pred_masks.cpu() # (1, 1, num_cands, H_low, W_low)
|
| 117 |
+
iou_scores = outputs.iou_scores.cpu().numpy()[0, 0]
|
| 118 |
+
best_idx = int(np.argmax(iou_scores))
|
| 119 |
+
best_score = float(iou_scores[best_idx])
|
| 120 |
+
|
| 121 |
+
if upscale_mode == "soft":
|
| 122 |
+
logits = pred_masks[0, 0, best_idx].to(torch.float32) # (H_low, W_low)
|
| 123 |
+
logits_4d = logits.unsqueeze(0).unsqueeze(0)
|
| 124 |
+
upsampled = F.interpolate(
|
| 125 |
+
logits_4d,
|
| 126 |
+
size=(h_full, w_full),
|
| 127 |
+
mode="bilinear",
|
| 128 |
+
align_corners=False,
|
| 129 |
+
)[0, 0].numpy()
|
| 130 |
+
mask_full = upsampled > 0.0
|
| 131 |
+
else:
|
| 132 |
+
masks_scaled = processor.post_process_masks(
|
| 133 |
+
outputs.pred_masks.cpu(),
|
| 134 |
+
inputs["original_sizes"],
|
| 135 |
+
mask_threshold=0.0,
|
| 136 |
+
)[0][0]
|
| 137 |
+
mask_scaled = masks_scaled[best_idx].numpy().astype(np.uint8) # scaled-res
|
| 138 |
+
|
| 139 |
+
if mask_scaled.shape != (h_full, w_full):
|
| 140 |
+
if upscale_mode == "nearest_hard":
|
| 141 |
+
interp = cv2.INTER_NEAREST
|
| 142 |
+
elif upscale_mode == "linear_hard":
|
| 143 |
+
interp = cv2.INTER_LINEAR
|
| 144 |
+
else:
|
| 145 |
+
raise ValueError(f"unknown upscale_mode: {upscale_mode}")
|
| 146 |
+
resized = cv2.resize(mask_scaled, (w_full, h_full), interpolation=interp)
|
| 147 |
+
if upscale_mode == "linear_hard":
|
| 148 |
+
mask_full = resized >= 1 # rethreshold after linear interp
|
| 149 |
+
else:
|
| 150 |
+
mask_full = resized.astype(bool)
|
| 151 |
+
else:
|
| 152 |
+
mask_full = mask_scaled.astype(bool)
|
| 153 |
+
|
| 154 |
+
return mask_full, best_score, time.time() - t0
|
| 155 |
+
|
| 156 |
+
|
| 157 |
+
# ----------------------------------------------------------------------------
|
| 158 |
+
# Metrics + visualization helpers
|
| 159 |
+
# ----------------------------------------------------------------------------
|
| 160 |
+
|
| 161 |
+
def _roughness_metrics(mask: np.ndarray) -> Dict[str, float]:
|
| 162 |
+
"""Perimeter + isoperimetric ratio of the largest contour."""
|
| 163 |
+
mask_u8 = (mask.astype(np.uint8)) * 255
|
| 164 |
+
contours, _ = cv2.findContours(mask_u8, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
|
| 165 |
+
if not contours:
|
| 166 |
+
return {"perimeter_px": 0.0, "area_px": 0.0, "iso_ratio": float("nan")}
|
| 167 |
+
largest = max(contours, key=cv2.contourArea)
|
| 168 |
+
perim = float(cv2.arcLength(largest, closed=True))
|
| 169 |
+
area = float(cv2.contourArea(largest))
|
| 170 |
+
iso = perim / (np.sqrt(area) + 1e-9)
|
| 171 |
+
return {"perimeter_px": perim, "area_px": area, "iso_ratio": iso}
|
| 172 |
+
|
| 173 |
+
|
| 174 |
+
def _save_overlay(
|
| 175 |
+
path: Path,
|
| 176 |
+
image_bgr: np.ndarray,
|
| 177 |
+
mask: np.ndarray,
|
| 178 |
+
palm_xy: Tuple[int, int],
|
| 179 |
+
label: str,
|
| 180 |
+
) -> None:
|
| 181 |
+
overlay = image_bgr.copy()
|
| 182 |
+
tint = np.zeros_like(overlay)
|
| 183 |
+
tint[mask] = (0, 255, 255)
|
| 184 |
+
overlay = cv2.addWeighted(overlay, 1.0, tint, 0.35, 0)
|
| 185 |
+
|
| 186 |
+
contours, _ = cv2.findContours(
|
| 187 |
+
(mask.astype(np.uint8)) * 255, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE
|
| 188 |
+
)
|
| 189 |
+
cv2.drawContours(overlay, contours, -1, (0, 255, 255), 2, cv2.LINE_AA)
|
| 190 |
+
|
| 191 |
+
cv2.circle(overlay, palm_xy, 18, (0, 255, 0), -1)
|
| 192 |
+
cv2.circle(overlay, palm_xy, 18, (0, 0, 0), 3)
|
| 193 |
+
|
| 194 |
+
cv2.putText(overlay, label, (30, 60), cv2.FONT_HERSHEY_SIMPLEX, 1.1,
|
| 195 |
+
(255, 255, 255), 5, cv2.LINE_AA)
|
| 196 |
+
cv2.putText(overlay, label, (30, 60), cv2.FONT_HERSHEY_SIMPLEX, 1.1,
|
| 197 |
+
(0, 255, 255), 2, cv2.LINE_AA)
|
| 198 |
+
cv2.imwrite(str(path), overlay)
|
| 199 |
+
|
| 200 |
+
|
| 201 |
+
def _save_fingertip_crop(
|
| 202 |
+
path: Path,
|
| 203 |
+
image_bgr: np.ndarray,
|
| 204 |
+
mask: np.ndarray,
|
| 205 |
+
center_xy: Tuple[int, int],
|
| 206 |
+
crop_half: int = 300,
|
| 207 |
+
label: str = "",
|
| 208 |
+
) -> None:
|
| 209 |
+
h, w = image_bgr.shape[:2]
|
| 210 |
+
cx, cy = center_xy
|
| 211 |
+
x0 = max(0, cx - crop_half)
|
| 212 |
+
y0 = max(0, cy - crop_half)
|
| 213 |
+
x1 = min(w, cx + crop_half)
|
| 214 |
+
y1 = min(h, cy + crop_half)
|
| 215 |
+
|
| 216 |
+
crop = image_bgr[y0:y1, x0:x1].copy()
|
| 217 |
+
mask_crop = mask[y0:y1, x0:x1]
|
| 218 |
+
|
| 219 |
+
tint = np.zeros_like(crop)
|
| 220 |
+
tint[mask_crop] = (0, 255, 255)
|
| 221 |
+
crop = cv2.addWeighted(crop, 1.0, tint, 0.4, 0)
|
| 222 |
+
|
| 223 |
+
contours, _ = cv2.findContours(
|
| 224 |
+
(mask_crop.astype(np.uint8)) * 255, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE
|
| 225 |
+
)
|
| 226 |
+
cv2.drawContours(crop, contours, -1, (0, 255, 255), 2, cv2.LINE_AA)
|
| 227 |
+
|
| 228 |
+
if label:
|
| 229 |
+
cv2.putText(crop, label, (15, 35), cv2.FONT_HERSHEY_SIMPLEX, 0.9,
|
| 230 |
+
(255, 255, 255), 4, cv2.LINE_AA)
|
| 231 |
+
cv2.putText(crop, label, (15, 35), cv2.FONT_HERSHEY_SIMPLEX, 0.9,
|
| 232 |
+
(0, 255, 255), 2, cv2.LINE_AA)
|
| 233 |
+
|
| 234 |
+
cv2.imwrite(str(path), crop)
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
# ----------------------------------------------------------------------------
|
| 238 |
+
# Main
|
| 239 |
+
# ----------------------------------------------------------------------------
|
| 240 |
+
|
| 241 |
+
def main() -> int:
|
| 242 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 243 |
+
parser.add_argument("--input", default="input/sample-04-12/card_2.jpg")
|
| 244 |
+
parser.add_argument("--output-dir", default="output/sam_mask_quality")
|
| 245 |
+
args = parser.parse_args()
|
| 246 |
+
|
| 247 |
+
in_path = Path(args.input)
|
| 248 |
+
out_dir = Path(args.output_dir)
|
| 249 |
+
out_dir.mkdir(parents=True, exist_ok=True)
|
| 250 |
+
|
| 251 |
+
image = cv2.imread(str(in_path))
|
| 252 |
+
if image is None:
|
| 253 |
+
print(f"ERROR: could not read {in_path}")
|
| 254 |
+
return 2
|
| 255 |
+
|
| 256 |
+
print(f"Input: {in_path} ({image.shape[1]}x{image.shape[0]})")
|
| 257 |
+
|
| 258 |
+
# Get canonical image + landmarks WITHOUT running SAM. Pass a large
|
| 259 |
+
# max_dimension so MediaPipe's internal resize does not happen and the
|
| 260 |
+
# canonical image stays at native resolution.
|
| 261 |
+
hand_data = segment_hand(
|
| 262 |
+
image=image,
|
| 263 |
+
finger="middle",
|
| 264 |
+
max_dimension=3000,
|
| 265 |
+
debug_dir=None,
|
| 266 |
+
use_sam_mask=False,
|
| 267 |
+
)
|
| 268 |
+
if hand_data is None:
|
| 269 |
+
print("ERROR: hand detection failed")
|
| 270 |
+
return 2
|
| 271 |
+
|
| 272 |
+
canonical = hand_data["canonical_image"]
|
| 273 |
+
landmarks = hand_data["landmarks"] # (21, 2) in canonical px coords
|
| 274 |
+
palm_xy = palm_center_from_landmarks(landmarks)
|
| 275 |
+
middle_tip_xy = (int(round(landmarks[12, 0])), int(round(landmarks[12, 1])))
|
| 276 |
+
|
| 277 |
+
ch, cw = canonical.shape[:2]
|
| 278 |
+
print(f"Canonical: {cw}x{ch} palm=({palm_xy[0]},{palm_xy[1]}) "
|
| 279 |
+
f"middle_tip=({middle_tip_xy[0]},{middle_tip_xy[1]})")
|
| 280 |
+
|
| 281 |
+
# Save canonical reference image
|
| 282 |
+
cv2.imwrite(str(out_dir / "00_canonical.png"), canonical)
|
| 283 |
+
|
| 284 |
+
variants = [
|
| 285 |
+
("A_baseline_1024_nn", 1024, "nearest_hard"),
|
| 286 |
+
("B_1024_linear_hard", 1024, "linear_hard"),
|
| 287 |
+
("C_1024_soft", 1024, "soft"),
|
| 288 |
+
("D_native_nn", None, "nearest_hard"),
|
| 289 |
+
("E_native_soft", None, "soft"),
|
| 290 |
+
]
|
| 291 |
+
|
| 292 |
+
results = []
|
| 293 |
+
for name, long_side, mode in variants:
|
| 294 |
+
print(f"\n=== {name} long_side={long_side} mode={mode} ===")
|
| 295 |
+
mask, score, secs = _run_sam(canonical, palm_xy, long_side, mode)
|
| 296 |
+
m = _roughness_metrics(mask)
|
| 297 |
+
print(f" iou={score:.3f} time={secs:.2f}s "
|
| 298 |
+
f"perim={m['perimeter_px']:.0f}px iso={m['iso_ratio']:.3f}")
|
| 299 |
+
|
| 300 |
+
cv2.imwrite(str(out_dir / f"{name}_mask.png"),
|
| 301 |
+
(mask.astype(np.uint8)) * 255)
|
| 302 |
+
_save_overlay(
|
| 303 |
+
out_dir / f"{name}_overlay.png",
|
| 304 |
+
canonical, mask, palm_xy,
|
| 305 |
+
label=f"{name} iou={score:.2f}",
|
| 306 |
+
)
|
| 307 |
+
_save_fingertip_crop(
|
| 308 |
+
out_dir / f"{name}_fingertip.png",
|
| 309 |
+
canonical, mask, middle_tip_xy,
|
| 310 |
+
crop_half=300,
|
| 311 |
+
label=name,
|
| 312 |
+
)
|
| 313 |
+
|
| 314 |
+
results.append((name, score, secs, m["perimeter_px"], m["iso_ratio"]))
|
| 315 |
+
|
| 316 |
+
# Summary table
|
| 317 |
+
print("\n")
|
| 318 |
+
print("=" * 78)
|
| 319 |
+
print(f"{'variant':<22}{'iou':>8}{'time(s)':>10}"
|
| 320 |
+
f"{'perim(px)':>14}{'iso':>10}{'vs A (%)':>12}")
|
| 321 |
+
print("-" * 78)
|
| 322 |
+
iso_base = results[0][4]
|
| 323 |
+
for name, score, secs, perim, iso in results:
|
| 324 |
+
rel = (iso / iso_base - 1.0) * 100.0 if iso_base else float("nan")
|
| 325 |
+
print(f"{name:<22}{score:>8.3f}{secs:>10.2f}"
|
| 326 |
+
f"{perim:>14.0f}{iso:>10.3f}{rel:>11.1f}%")
|
| 327 |
+
print("=" * 78)
|
| 328 |
+
|
| 329 |
+
# Side-by-side fingertip comparison
|
| 330 |
+
crops = [cv2.imread(str(out_dir / f"{name}_fingertip.png")) for name, *_ in results]
|
| 331 |
+
if all(c is not None for c in crops):
|
| 332 |
+
panel = np.hstack(crops)
|
| 333 |
+
cv2.imwrite(str(out_dir / "fingertip_comparison.png"), panel)
|
| 334 |
+
print(f"\nFingertip comparison strip: {out_dir / 'fingertip_comparison.png'}")
|
| 335 |
+
|
| 336 |
+
print(f"\nAll outputs saved to: {out_dir}/")
|
| 337 |
+
return 0
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
if __name__ == "__main__":
|
| 341 |
+
raise SystemExit(main())
|
src/debug_observer.py
CHANGED
|
@@ -1160,9 +1160,6 @@ def draw_comprehensive_edge_overlay(
|
|
| 1160 |
p2 = (zone_pt - perp_direction * roi_half_width).astype(int)
|
| 1161 |
cv2.line(vis, tuple(p1), tuple(p2), Color.ORANGE, 2, cv2.LINE_AA)
|
| 1162 |
|
| 1163 |
-
# 3. Draw ROI boundary
|
| 1164 |
-
cv2.rectangle(vis, (x_min, y_min), (x_max, y_max), Color.CYAN, 2)
|
| 1165 |
-
|
| 1166 |
# 4. Draw detected edges
|
| 1167 |
line_spacing = max(1, int(np.sum(valid_rows)) // 25) # Show ~25 lines
|
| 1168 |
count = 0
|
|
@@ -1221,7 +1218,6 @@ def draw_comprehensive_edge_overlay(
|
|
| 1221 |
"Legend:",
|
| 1222 |
" Yellow line = Finger axis",
|
| 1223 |
" Orange lines = Ring zone",
|
| 1224 |
-
" Cyan box = ROI",
|
| 1225 |
" Blue dots = Left edges",
|
| 1226 |
" Magenta dots = Right edges",
|
| 1227 |
" Green lines = Width measurements"
|
|
|
|
| 1160 |
p2 = (zone_pt - perp_direction * roi_half_width).astype(int)
|
| 1161 |
cv2.line(vis, tuple(p1), tuple(p2), Color.ORANGE, 2, cv2.LINE_AA)
|
| 1162 |
|
|
|
|
|
|
|
|
|
|
| 1163 |
# 4. Draw detected edges
|
| 1164 |
line_spacing = max(1, int(np.sum(valid_rows)) // 25) # Show ~25 lines
|
| 1165 |
count = 0
|
|
|
|
| 1218 |
"Legend:",
|
| 1219 |
" Yellow line = Finger axis",
|
| 1220 |
" Orange lines = Ring zone",
|
|
|
|
| 1221 |
" Blue dots = Left edges",
|
| 1222 |
" Magenta dots = Right edges",
|
| 1223 |
" Green lines = Width measurements"
|
src/sam_hand_segmentation.py
CHANGED
|
@@ -59,6 +59,7 @@ def segment_hand_sam(
|
|
| 59 |
Bool mask of the same shape as `image_bgr[:2]`, or None on failure.
|
| 60 |
"""
|
| 61 |
import torch
|
|
|
|
| 62 |
from PIL import Image as PILImage
|
| 63 |
|
| 64 |
h_full, w_full = image_bgr.shape[:2]
|
|
@@ -89,26 +90,25 @@ def segment_hand_sam(
|
|
| 89 |
with torch.inference_mode():
|
| 90 |
outputs = model(**inputs, multimask_output=True)
|
| 91 |
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
|
|
|
| 97 |
scores = outputs.iou_scores.cpu().numpy()[0, 0]
|
| 98 |
best_idx = int(np.argmax(scores))
|
| 99 |
-
mask_scaled = masks[best_idx].numpy().astype(bool)
|
| 100 |
best_score = float(scores[best_idx])
|
| 101 |
-
infer_time = time.time() - t0
|
| 102 |
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
|
| 113 |
print(
|
| 114 |
f" SAM hand mask: score={best_score:.3f} time={infer_time:.1f}s "
|
|
|
|
| 59 |
Bool mask of the same shape as `image_bgr[:2]`, or None on failure.
|
| 60 |
"""
|
| 61 |
import torch
|
| 62 |
+
import torch.nn.functional as F
|
| 63 |
from PIL import Image as PILImage
|
| 64 |
|
| 65 |
h_full, w_full = image_bgr.shape[:2]
|
|
|
|
| 90 |
with torch.inference_mode():
|
| 91 |
outputs = model(**inputs, multimask_output=True)
|
| 92 |
|
| 93 |
+
# Use the raw 256x256 logits and bilinearly upsample them to the full
|
| 94 |
+
# resolution before thresholding. Going through post_process_masks +
|
| 95 |
+
# cv2.INTER_NEAREST binarizes at the scaled resolution and then blows
|
| 96 |
+
# the hard mask up ~4x, which produces visible staircase edges on the
|
| 97 |
+
# finger boundaries (see script/experiment_sam_mask_quality.py).
|
| 98 |
+
pred_masks = outputs.pred_masks.cpu() # (1, 1, num_cands, H_low, W_low)
|
| 99 |
scores = outputs.iou_scores.cpu().numpy()[0, 0]
|
| 100 |
best_idx = int(np.argmax(scores))
|
|
|
|
| 101 |
best_score = float(scores[best_idx])
|
|
|
|
| 102 |
|
| 103 |
+
logits_best = pred_masks[0, 0, best_idx].to(torch.float32)
|
| 104 |
+
upsampled = F.interpolate(
|
| 105 |
+
logits_best.unsqueeze(0).unsqueeze(0),
|
| 106 |
+
size=(h_full, w_full),
|
| 107 |
+
mode="bilinear",
|
| 108 |
+
align_corners=False,
|
| 109 |
+
)[0, 0].numpy()
|
| 110 |
+
mask_full = upsampled > 0.0
|
| 111 |
+
infer_time = time.time() - t0
|
| 112 |
|
| 113 |
print(
|
| 114 |
f" SAM hand mask: score={best_score:.3f} time={infer_time:.1f}s "
|