Spaces:
Runtime error
Runtime error
Switch rooftop masking to SAM3 and refresh demos
Browse files- ARCHITECTURE.md +5 -5
- README.md +1 -1
- app/config.py +7 -4
- app/depth_pipeline.py +0 -23
- app/safety.py +18 -67
- app/segmentation.py +9 -1
- app/ui.py +28 -15
- demo/__init__.py +1 -0
- demo/curated.py +24 -27
- demo/curated_ui.py +3 -6
- demo/demo_app.py +56 -0
- demo/precompute_curated.py +5 -5
ARCHITECTURE.md
CHANGED
|
@@ -5,7 +5,7 @@ This document describes the flow in the current Gradio app (`app/ui.py`), from i
|
|
| 5 |
## Data and Models
|
| 6 |
- **Inputs**: Images under `data/Image/` (VISLOC and any custom folders) via `list_all_data_inputs`, with a 5% border crop (`crop_nonblack`) to drop black padding. Supported extensions: jpg/jpeg/png (any case).
|
| 7 |
- **Depth model**: Depth Anything 3, cached per model id (`DepthEngine`). Inference caps the long side to `process_res_cap` (default 1024) using `upper_bound_resize` before predicting.
|
| 8 |
-
- **Segmentation model**: SAM3 (`facebook/sam3`) for promptable water/road/tree masking. The segmenter is cached per model id but masks are recomputed every run (no output cache). Default `segmentation_max_side` is
|
| 9 |
|
| 10 |
## Constants and Defaults
|
| 11 |
- Altitude/FOV defaults: 450 m, 90° (footprint default 10 m).
|
|
@@ -14,7 +14,7 @@ This document describes the flow in the current Gradio app (`app/ui.py`), from i
|
|
| 14 |
- Coverage strictness: default 0.95 (fraction of the footprint that must be safe).
|
| 15 |
- Texture threshold: default 0.3 (suppresses highly textured regions).
|
| 16 |
- Depth smoothing is supported but set to 0.0 in the UI (effectively off).
|
| 17 |
-
- Roof mask:
|
| 18 |
|
| 19 |
## Per-Image Processing Pipeline
|
| 20 |
1. **Load and crop** the selected image (RGB, 5% border removed).
|
|
@@ -25,8 +25,7 @@ This document describes the flow in the current Gradio app (`app/ui.py`), from i
|
|
| 25 |
- Visualization window `vis_patch` is an odd size capped to 1/8 of the smallest depth dimension for sharper std previews.
|
| 26 |
4. **Texture mask**: Sobel magnitude on the RGB (blurred by `patch_px/40`), normalized; pixels above `texture_threshold` are suppressed.
|
| 27 |
5. **Segmentation masks (optional)**:
|
| 28 |
-
- Water/Road/Tree via SAM3 at `segmentation_max_side`, with text prompts. Instance masks are unioned per class, resized to depth scale, and dilated to footprint size for blocking.
|
| 29 |
-
- Roof via depth MAD threshold + grad gate; components smaller than `area_thresh` and not exceeding `max_area_frac` (20%) are kept; also dilated to the footprint for blocking.
|
| 30 |
6. **Flat region search (`pick_flat_patch`)**:
|
| 31 |
- Normalize depth to [0,1], compute `std_map` via box mean/mean_sq, and `grad_norm` via `np.gradient` normalized at the 95th percentile.
|
| 32 |
- Landing mask starts from `grad_norm < grad_thresh_eff`, excludes water if present, and keeps the lowest-variance patch as a fallback box.
|
|
@@ -72,9 +71,10 @@ This document describes the flow in the current Gradio app (`app/ui.py`), from i
|
|
| 72 |
|
| 73 |
## Error Handling
|
| 74 |
- Bad/missing inputs raise Gradio errors.
|
| 75 |
-
- Segmentation failures warn and proceed without that mask;
|
| 76 |
- Coverage/boxFilter fallbacks keep processing even if OpenCV operations fail.
|
| 77 |
|
| 78 |
## Outputs
|
| 79 |
- Dict of PIL Images keyed by: RGB, Depth, Flatness map (std), Depth gradient, Gradient mask, Water mask, Road mask, Tree mask, Roof mask, Safety heatmap overlay, Hazard overlay, Water/Road/Tree hazard overlays, Flatness heatmap overlay, Safety score (grayscale), Landing spot overlay.
|
|
|
|
| 80 |
- `compose_view` uses these to build the preview.
|
|
|
|
| 5 |
## Data and Models
|
| 6 |
- **Inputs**: Images under `data/Image/` (VISLOC and any custom folders) via `list_all_data_inputs`, with a 5% border crop (`crop_nonblack`) to drop black padding. Supported extensions: jpg/jpeg/png (any case).
|
| 7 |
- **Depth model**: Depth Anything 3, cached per model id (`DepthEngine`). Inference caps the long side to `process_res_cap` (default 1024) using `upper_bound_resize` before predicting.
|
| 8 |
+
- **Segmentation model**: SAM3 (`facebook/sam3`) for promptable water/road/tree/roof masking. The segmenter is cached per model id but masks are recomputed every run (no output cache). Default `segmentation_max_side` is 512 and is clamped to the depth resolution (min 128).
|
| 9 |
|
| 10 |
## Constants and Defaults
|
| 11 |
- Altitude/FOV defaults: 450 m, 90° (footprint default 10 m).
|
|
|
|
| 14 |
- Coverage strictness: default 0.95 (fraction of the footprint that must be safe).
|
| 15 |
- Texture threshold: default 0.3 (suppresses highly textured regions).
|
| 16 |
- Depth smoothing is supported but set to 0.0 in the UI (effectively off).
|
| 17 |
+
- Roof mask: SAM3 promptable segmentation (default prompt: `roof`), resized to depth scale and expanded to footprint size; no depth-based roof heuristics remain.
|
| 18 |
|
| 19 |
## Per-Image Processing Pipeline
|
| 20 |
1. **Load and crop** the selected image (RGB, 5% border removed).
|
|
|
|
| 25 |
- Visualization window `vis_patch` is an odd size capped to 1/8 of the smallest depth dimension for sharper std previews.
|
| 26 |
4. **Texture mask**: Sobel magnitude on the RGB (blurred by `patch_px/40`), normalized; pixels above `texture_threshold` are suppressed.
|
| 27 |
5. **Segmentation masks (optional)**:
|
| 28 |
+
- Water/Road/Tree/Roof via SAM3 at `segmentation_max_side`, with text prompts. Instance masks are unioned per class, resized to depth scale, and dilated to footprint size for blocking.
|
|
|
|
| 29 |
6. **Flat region search (`pick_flat_patch`)**:
|
| 30 |
- Normalize depth to [0,1], compute `std_map` via box mean/mean_sq, and `grad_norm` via `np.gradient` normalized at the 95th percentile.
|
| 31 |
- Landing mask starts from `grad_norm < grad_thresh_eff`, excludes water if present, and keeps the lowest-variance patch as a fallback box.
|
|
|
|
| 71 |
|
| 72 |
## Error Handling
|
| 73 |
- Bad/missing inputs raise Gradio errors.
|
| 74 |
+
- Segmentation failures warn and proceed without that mask; water/road/tree/roof warnings clarify when masks are disabled or not detected.
|
| 75 |
- Coverage/boxFilter fallbacks keep processing even if OpenCV operations fail.
|
| 76 |
|
| 77 |
## Outputs
|
| 78 |
- Dict of PIL Images keyed by: RGB, Depth, Flatness map (std), Depth gradient, Gradient mask, Water mask, Road mask, Tree mask, Roof mask, Safety heatmap overlay, Hazard overlay, Water/Road/Tree hazard overlays, Flatness heatmap overlay, Safety score (grayscale), Landing spot overlay.
|
| 79 |
+
- Run summaries surface model id, process resolution, runtime, footprint size (depth + image scale), landing center, safe/hazard coverage, effective thresholds, per-mask coverage (water/road/tree/roof), and warnings for disabled/absent masks or missing safe regions; the UI cards render these fields directly.
|
| 80 |
- `compose_view` uses these to build the preview.
|
README.md
CHANGED
|
@@ -10,7 +10,7 @@ Analyze aerial RGB imagery to detect safe drone landing sites. Combines monocula
|
|
| 10 |
|
| 11 |
## What’s inside
|
| 12 |
- **Main app (`landing_app.py`)** — runs full inference with adjustable thresholds, overlays, and camera assumptions; requires >8GB VRAM (assuming default 1024 px processing resolution); runtime is ~1000ms per image.
|
| 13 |
-
- **Curated gallery (`demo/
|
| 14 |
|
| 15 |
## Prereqs
|
| 16 |
- Python 3.10+ and a CUDA GPU for the main app (CPU works but is slow).
|
|
|
|
| 10 |
|
| 11 |
## What’s inside
|
| 12 |
- **Main app (`landing_app.py`)** — runs full inference with adjustable thresholds, overlays, and camera assumptions; requires >8GB VRAM (assuming default 1024 px processing resolution); runtime is ~1000ms per image.
|
| 13 |
+
- **Curated gallery (`demo/demo_app.py`)** — precomputed PNG/JPG/JSON artifacts for fast, zero-GPU browsing.
|
| 14 |
|
| 15 |
## Prereqs
|
| 16 |
- Python 3.10+ and a CUDA GPU for the main app (CPU works but is slow).
|
app/config.py
CHANGED
|
@@ -10,12 +10,13 @@ DEFAULT_ALTITUDE_M = 450.0
|
|
| 10 |
ASSUMED_FOV_DEG = 90.0
|
| 11 |
DEFAULT_MODEL_ID = "depth-anything/DA3MONO-LARGE"
|
| 12 |
SEGMENTATION_MODEL_ID = "facebook/sam3"
|
| 13 |
-
SEGMENTATION_MAX_SIDE =
|
| 14 |
SEGMENTATION_SCORE_THRESH = 0.25
|
| 15 |
SEGMENTATION_MASK_THRESH = 0.25
|
| 16 |
WATER_PROMPT = "water"
|
| 17 |
ROAD_PROMPT = "motorway"
|
| 18 |
TREE_PROMPT = "trees"
|
|
|
|
| 19 |
|
| 20 |
|
| 21 |
@dataclass(frozen=True)
|
|
@@ -25,7 +26,7 @@ class AnalyzerSettings:
|
|
| 25 |
footprint_m: float = 10.0
|
| 26 |
std_thresh: float = 0.005
|
| 27 |
grad_thresh: float = 0.1
|
| 28 |
-
clearance_factor: float = 1.
|
| 29 |
process_res_cap: int = 1024
|
| 30 |
depth_smoothing_base: float = 0.0
|
| 31 |
segmentation_max_side: int = SEGMENTATION_MAX_SIDE
|
|
@@ -35,9 +36,10 @@ class AnalyzerSettings:
|
|
| 35 |
water_prompt: str = WATER_PROMPT
|
| 36 |
road_prompt: str = ROAD_PROMPT
|
| 37 |
tree_prompt: str = TREE_PROMPT
|
|
|
|
| 38 |
coverage_strictness: float = 0.95
|
| 39 |
-
openness_weight: float = 0.
|
| 40 |
-
texture_threshold: float = 0.
|
| 41 |
altitude_m: float = DEFAULT_ALTITUDE_M
|
| 42 |
fov_deg: float = ASSUMED_FOV_DEG
|
| 43 |
model_id: str = DEFAULT_MODEL_ID
|
|
@@ -60,6 +62,7 @@ __all__ = [
|
|
| 60 |
"WATER_PROMPT",
|
| 61 |
"ROAD_PROMPT",
|
| 62 |
"TREE_PROMPT",
|
|
|
|
| 63 |
"DEFAULT_ANALYZER_SETTINGS",
|
| 64 |
"AnalyzerSettings",
|
| 65 |
]
|
|
|
|
| 10 |
ASSUMED_FOV_DEG = 90.0
|
| 11 |
DEFAULT_MODEL_ID = "depth-anything/DA3MONO-LARGE"
|
| 12 |
SEGMENTATION_MODEL_ID = "facebook/sam3"
|
| 13 |
+
SEGMENTATION_MAX_SIDE = 512
|
| 14 |
SEGMENTATION_SCORE_THRESH = 0.25
|
| 15 |
SEGMENTATION_MASK_THRESH = 0.25
|
| 16 |
WATER_PROMPT = "water"
|
| 17 |
ROAD_PROMPT = "motorway"
|
| 18 |
TREE_PROMPT = "trees"
|
| 19 |
+
ROOF_PROMPT = "rooftop"
|
| 20 |
|
| 21 |
|
| 22 |
@dataclass(frozen=True)
|
|
|
|
| 26 |
footprint_m: float = 10.0
|
| 27 |
std_thresh: float = 0.005
|
| 28 |
grad_thresh: float = 0.1
|
| 29 |
+
clearance_factor: float = 1.5
|
| 30 |
process_res_cap: int = 1024
|
| 31 |
depth_smoothing_base: float = 0.0
|
| 32 |
segmentation_max_side: int = SEGMENTATION_MAX_SIDE
|
|
|
|
| 36 |
water_prompt: str = WATER_PROMPT
|
| 37 |
road_prompt: str = ROAD_PROMPT
|
| 38 |
tree_prompt: str = TREE_PROMPT
|
| 39 |
+
roof_prompt: str = ROOF_PROMPT
|
| 40 |
coverage_strictness: float = 0.95
|
| 41 |
+
openness_weight: float = 0.5
|
| 42 |
+
texture_threshold: float = 0.5
|
| 43 |
altitude_m: float = DEFAULT_ALTITUDE_M
|
| 44 |
fov_deg: float = ASSUMED_FOV_DEG
|
| 45 |
model_id: str = DEFAULT_MODEL_ID
|
|
|
|
| 62 |
"WATER_PROMPT",
|
| 63 |
"ROAD_PROMPT",
|
| 64 |
"TREE_PROMPT",
|
| 65 |
+
"ROOF_PROMPT",
|
| 66 |
"DEFAULT_ANALYZER_SETTINGS",
|
| 67 |
"AnalyzerSettings",
|
| 68 |
]
|
app/depth_pipeline.py
CHANGED
|
@@ -1,9 +1,6 @@
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
| 3 |
-
import functools
|
| 4 |
-
import math
|
| 5 |
from pathlib import Path
|
| 6 |
-
from typing import Tuple
|
| 7 |
|
| 8 |
import cv2
|
| 9 |
import numpy as np
|
|
@@ -29,25 +26,6 @@ def crop_nonblack(img: Image.Image, frac: float = 0.05) -> Image.Image:
|
|
| 29 |
return img.crop((dx, dy, w - dx, h - dy))
|
| 30 |
|
| 31 |
|
| 32 |
-
def compute_roof_mask_depth(depth: np.ndarray, aggressiveness: float = 1.3, morph_kernel: int = 5) -> np.ndarray:
|
| 33 |
-
d = depth.astype(np.float32)
|
| 34 |
-
med = np.median(d)
|
| 35 |
-
mad = np.median(np.abs(d - med)) + 1e-6
|
| 36 |
-
threshold = med - aggressiveness * mad
|
| 37 |
-
mask = d < threshold
|
| 38 |
-
mask = mask.astype(np.uint8)
|
| 39 |
-
k = max(1, int(morph_kernel))
|
| 40 |
-
if k % 2 == 0:
|
| 41 |
-
k += 1
|
| 42 |
-
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (k, k))
|
| 43 |
-
try:
|
| 44 |
-
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
|
| 45 |
-
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
|
| 46 |
-
except Exception:
|
| 47 |
-
pass
|
| 48 |
-
return mask > 0
|
| 49 |
-
|
| 50 |
-
|
| 51 |
def remove_global_plane(depth: np.ndarray, method: str = "least_squares") -> np.ndarray:
|
| 52 |
if depth.ndim != 2:
|
| 53 |
return depth
|
|
@@ -182,7 +160,6 @@ def smooth_depth(depth: np.ndarray, sigma: float) -> np.ndarray:
|
|
| 182 |
|
| 183 |
__all__ = [
|
| 184 |
"DepthEngine",
|
| 185 |
-
"compute_roof_mask_depth",
|
| 186 |
"crop_nonblack",
|
| 187 |
"pick_flat_patch",
|
| 188 |
"remove_global_plane",
|
|
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
|
|
|
|
|
|
| 3 |
from pathlib import Path
|
|
|
|
| 4 |
|
| 5 |
import cv2
|
| 6 |
import numpy as np
|
|
|
|
| 26 |
return img.crop((dx, dy, w - dx, h - dy))
|
| 27 |
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
def remove_global_plane(depth: np.ndarray, method: str = "least_squares") -> np.ndarray:
|
| 30 |
if depth.ndim != 2:
|
| 31 |
return depth
|
|
|
|
| 160 |
|
| 161 |
__all__ = [
|
| 162 |
"DepthEngine",
|
|
|
|
| 163 |
"crop_nonblack",
|
| 164 |
"pick_flat_patch",
|
| 165 |
"remove_global_plane",
|
app/safety.py
CHANGED
|
@@ -11,7 +11,7 @@ import torch
|
|
| 11 |
from PIL import Image
|
| 12 |
|
| 13 |
from .config import DEFAULT_MODEL_ID, IMAGE_EXTS
|
| 14 |
-
from .depth_pipeline import DepthEngine,
|
| 15 |
from .segmentation import SegmenterRequest, SegmenterService, get_global_segmenter
|
| 16 |
from .visualization import build_result_layers
|
| 17 |
|
|
@@ -27,6 +27,7 @@ class AnalysisRequest:
|
|
| 27 |
use_tree_mask: bool
|
| 28 |
water_prompt: str
|
| 29 |
road_prompt: str
|
|
|
|
| 30 |
tree_prompt: str
|
| 31 |
altitude_m: float
|
| 32 |
fov_deg: float
|
|
@@ -59,9 +60,11 @@ class AnalysisSummary:
|
|
| 59 |
water_mask_pct: Optional[float]
|
| 60 |
road_mask_pct: Optional[float]
|
| 61 |
roof_mask_pct: Optional[float]
|
|
|
|
| 62 |
water_mask_enabled: bool
|
| 63 |
road_mask_enabled: bool
|
| 64 |
roof_mask_enabled: bool
|
|
|
|
| 65 |
used_valid_center: bool
|
| 66 |
warnings: list[str]
|
| 67 |
std_thresh_applied: float
|
|
@@ -84,64 +87,6 @@ class SafetyAnalyzer:
|
|
| 84 |
except Exception as exc:
|
| 85 |
print(f"[WARN] Could not preload depth model {DEFAULT_MODEL_ID}: {exc}")
|
| 86 |
|
| 87 |
-
@staticmethod
|
| 88 |
-
def build_depth_roof_mask(
|
| 89 |
-
depth: np.ndarray,
|
| 90 |
-
grad_norm: np.ndarray,
|
| 91 |
-
footprint_px: int,
|
| 92 |
-
aggressiveness: float = 1.2,
|
| 93 |
-
grad_threshold: float = 0.35,
|
| 94 |
-
max_area_frac: float = 0.2,
|
| 95 |
-
) -> np.ndarray | None:
|
| 96 |
-
depth_mask = compute_roof_mask_depth(
|
| 97 |
-
depth,
|
| 98 |
-
aggressiveness=aggressiveness,
|
| 99 |
-
morph_kernel=max(3, int(round(max(3, footprint_px * 0.15))) | 1),
|
| 100 |
-
)
|
| 101 |
-
flat_mask = grad_norm < grad_threshold
|
| 102 |
-
roof_mask = depth_mask & flat_mask
|
| 103 |
-
roof_mask = roof_mask.astype(np.uint8)
|
| 104 |
-
kernel = cv2.getStructuringElement(
|
| 105 |
-
cv2.MORPH_ELLIPSE,
|
| 106 |
-
(
|
| 107 |
-
max(3, int(round(footprint_px * 0.1)) | 1),
|
| 108 |
-
max(3, int(round(footprint_px * 0.1)) | 1),
|
| 109 |
-
),
|
| 110 |
-
)
|
| 111 |
-
roof_mask = cv2.morphologyEx(roof_mask, cv2.MORPH_CLOSE, kernel)
|
| 112 |
-
roof_mask = cv2.morphologyEx(roof_mask, cv2.MORPH_OPEN, kernel)
|
| 113 |
-
area_thresh = max(footprint_px * footprint_px // 4, 64)
|
| 114 |
-
num_labels, labels, stats, _ = cv2.connectedComponentsWithStats(roof_mask, connectivity=8)
|
| 115 |
-
refined = np.zeros_like(roof_mask, dtype=bool)
|
| 116 |
-
depth_norm = (depth - depth.min()) / (np.ptp(depth) + 1e-6)
|
| 117 |
-
global_mad = np.median(np.abs(depth_norm - np.median(depth_norm))) + 1e-6
|
| 118 |
-
ring_kernel = cv2.getStructuringElement(
|
| 119 |
-
cv2.MORPH_ELLIPSE,
|
| 120 |
-
(
|
| 121 |
-
max(3, int(round(footprint_px * 0.6)) | 1),
|
| 122 |
-
max(3, int(round(footprint_px * 0.6)) | 1),
|
| 123 |
-
),
|
| 124 |
-
)
|
| 125 |
-
max_area = max_area_frac * depth_mask.size if max_area_frac > 0 else None
|
| 126 |
-
for i in range(1, num_labels):
|
| 127 |
-
area = stats[i, cv2.CC_STAT_AREA]
|
| 128 |
-
if area < area_thresh:
|
| 129 |
-
continue
|
| 130 |
-
if max_area is not None and area > max_area:
|
| 131 |
-
# Skip overly large blobs (e.g., entire fields) to avoid over-masking
|
| 132 |
-
continue
|
| 133 |
-
comp_mask = labels == i
|
| 134 |
-
ring = cv2.dilate(comp_mask.astype(np.uint8), ring_kernel, iterations=1).astype(bool) & (~comp_mask)
|
| 135 |
-
if not ring.any():
|
| 136 |
-
continue
|
| 137 |
-
comp_mean = float(depth_norm[comp_mask].mean())
|
| 138 |
-
ring_mean = float(depth_norm[ring].mean())
|
| 139 |
-
prominence = ring_mean - comp_mean
|
| 140 |
-
min_prominence = max(0.02, 0.5 * global_mad)
|
| 141 |
-
if prominence < min_prominence or ring_mean <= comp_mean:
|
| 142 |
-
continue
|
| 143 |
-
refined |= comp_mask
|
| 144 |
-
return refined if refined.any() else None
|
| 145 |
|
| 146 |
def analyze_image(self, image: Image.Image, request: AnalysisRequest) -> AnalysisResult:
|
| 147 |
t0 = time.perf_counter()
|
|
@@ -238,10 +183,12 @@ class SafetyAnalyzer:
|
|
| 238 |
source_path=request.source_path,
|
| 239 |
want_water=request.use_water_mask,
|
| 240 |
want_road=request.use_road_mask,
|
|
|
|
| 241 |
want_tree=request.use_tree_mask,
|
| 242 |
max_side=int(max(128, min(request.segmentation_max_side, process_res))),
|
| 243 |
water_prompt=request.water_prompt,
|
| 244 |
road_prompt=request.road_prompt,
|
|
|
|
| 245 |
tree_prompt=request.tree_prompt,
|
| 246 |
score_threshold=float(request.segmentation_score_thresh),
|
| 247 |
mask_threshold=float(request.segmentation_mask_thresh),
|
|
@@ -260,6 +207,12 @@ class SafetyAnalyzer:
|
|
| 260 |
)
|
| 261 |
road_mask_resized = np.array(road_mask_resized) > 0
|
| 262 |
road_mask_block = expand_mask_for_footprint(road_mask_resized)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 263 |
if request.use_tree_mask and masks.get("tree") is not None:
|
| 264 |
tree_mask_resized = Image.fromarray(masks["tree"].astype(np.uint8) * 255).resize(
|
| 265 |
(depth.shape[1], depth.shape[0]), resample=Image.NEAREST
|
|
@@ -280,14 +233,6 @@ class SafetyAnalyzer:
|
|
| 280 |
water_mask=water_mask_block if water_mask_block is not None else water_mask_resized,
|
| 281 |
)
|
| 282 |
t_pick = time.perf_counter()
|
| 283 |
-
if request.use_roof_mask:
|
| 284 |
-
roof_mask_resized = self.build_depth_roof_mask(
|
| 285 |
-
depth=depth,
|
| 286 |
-
grad_norm=grad_norm,
|
| 287 |
-
footprint_px=patch_px,
|
| 288 |
-
max_area_frac=0.2,
|
| 289 |
-
)
|
| 290 |
-
roof_mask_block = expand_mask_for_footprint(roof_mask_resized)
|
| 291 |
seg_block_mask = None
|
| 292 |
for mask in (water_mask_block, road_mask_block, tree_mask_block, roof_mask_block):
|
| 293 |
if mask is None:
|
|
@@ -545,6 +490,10 @@ class SafetyAnalyzer:
|
|
| 545 |
warnings.append("Road mask disabled.")
|
| 546 |
elif road_mask_resized is None:
|
| 547 |
warnings.append("Road segmentation unavailable; continuing without mask.")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 548 |
if not request.use_roof_mask:
|
| 549 |
warnings.append("Roof mask disabled.")
|
| 550 |
elif roof_mask_resized is None:
|
|
@@ -578,9 +527,11 @@ class SafetyAnalyzer:
|
|
| 578 |
water_mask_pct=mask_pct(water_mask_resized) if request.use_water_mask else None,
|
| 579 |
road_mask_pct=mask_pct(road_mask_resized) if request.use_road_mask else None,
|
| 580 |
roof_mask_pct=mask_pct(roof_mask_resized) if request.use_roof_mask else None,
|
|
|
|
| 581 |
water_mask_enabled=request.use_water_mask,
|
| 582 |
road_mask_enabled=request.use_road_mask,
|
| 583 |
roof_mask_enabled=request.use_roof_mask,
|
|
|
|
| 584 |
used_valid_center=used_valid_center,
|
| 585 |
warnings=warnings,
|
| 586 |
std_thresh_applied=std_thresh_eff,
|
|
|
|
| 11 |
from PIL import Image
|
| 12 |
|
| 13 |
from .config import DEFAULT_MODEL_ID, IMAGE_EXTS
|
| 14 |
+
from .depth_pipeline import DepthEngine, crop_nonblack, pick_flat_patch, smooth_depth
|
| 15 |
from .segmentation import SegmenterRequest, SegmenterService, get_global_segmenter
|
| 16 |
from .visualization import build_result_layers
|
| 17 |
|
|
|
|
| 27 |
use_tree_mask: bool
|
| 28 |
water_prompt: str
|
| 29 |
road_prompt: str
|
| 30 |
+
roof_prompt: str
|
| 31 |
tree_prompt: str
|
| 32 |
altitude_m: float
|
| 33 |
fov_deg: float
|
|
|
|
| 60 |
water_mask_pct: Optional[float]
|
| 61 |
road_mask_pct: Optional[float]
|
| 62 |
roof_mask_pct: Optional[float]
|
| 63 |
+
tree_mask_pct: Optional[float]
|
| 64 |
water_mask_enabled: bool
|
| 65 |
road_mask_enabled: bool
|
| 66 |
roof_mask_enabled: bool
|
| 67 |
+
tree_mask_enabled: bool
|
| 68 |
used_valid_center: bool
|
| 69 |
warnings: list[str]
|
| 70 |
std_thresh_applied: float
|
|
|
|
| 87 |
except Exception as exc:
|
| 88 |
print(f"[WARN] Could not preload depth model {DEFAULT_MODEL_ID}: {exc}")
|
| 89 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
def analyze_image(self, image: Image.Image, request: AnalysisRequest) -> AnalysisResult:
|
| 92 |
t0 = time.perf_counter()
|
|
|
|
| 183 |
source_path=request.source_path,
|
| 184 |
want_water=request.use_water_mask,
|
| 185 |
want_road=request.use_road_mask,
|
| 186 |
+
want_roof=request.use_roof_mask,
|
| 187 |
want_tree=request.use_tree_mask,
|
| 188 |
max_side=int(max(128, min(request.segmentation_max_side, process_res))),
|
| 189 |
water_prompt=request.water_prompt,
|
| 190 |
road_prompt=request.road_prompt,
|
| 191 |
+
roof_prompt=request.roof_prompt,
|
| 192 |
tree_prompt=request.tree_prompt,
|
| 193 |
score_threshold=float(request.segmentation_score_thresh),
|
| 194 |
mask_threshold=float(request.segmentation_mask_thresh),
|
|
|
|
| 207 |
)
|
| 208 |
road_mask_resized = np.array(road_mask_resized) > 0
|
| 209 |
road_mask_block = expand_mask_for_footprint(road_mask_resized)
|
| 210 |
+
if request.use_roof_mask and masks.get("roof") is not None:
|
| 211 |
+
roof_mask_resized = Image.fromarray(masks["roof"].astype(np.uint8) * 255).resize(
|
| 212 |
+
(depth.shape[1], depth.shape[0]), resample=Image.NEAREST
|
| 213 |
+
)
|
| 214 |
+
roof_mask_resized = np.array(roof_mask_resized) > 0
|
| 215 |
+
roof_mask_block = expand_mask_for_footprint(roof_mask_resized)
|
| 216 |
if request.use_tree_mask and masks.get("tree") is not None:
|
| 217 |
tree_mask_resized = Image.fromarray(masks["tree"].astype(np.uint8) * 255).resize(
|
| 218 |
(depth.shape[1], depth.shape[0]), resample=Image.NEAREST
|
|
|
|
| 233 |
water_mask=water_mask_block if water_mask_block is not None else water_mask_resized,
|
| 234 |
)
|
| 235 |
t_pick = time.perf_counter()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 236 |
seg_block_mask = None
|
| 237 |
for mask in (water_mask_block, road_mask_block, tree_mask_block, roof_mask_block):
|
| 238 |
if mask is None:
|
|
|
|
| 490 |
warnings.append("Road mask disabled.")
|
| 491 |
elif road_mask_resized is None:
|
| 492 |
warnings.append("Road segmentation unavailable; continuing without mask.")
|
| 493 |
+
if not request.use_tree_mask:
|
| 494 |
+
warnings.append("Tree mask disabled.")
|
| 495 |
+
elif tree_mask_resized is None:
|
| 496 |
+
warnings.append("Tree segmentation unavailable; continuing without mask.")
|
| 497 |
if not request.use_roof_mask:
|
| 498 |
warnings.append("Roof mask disabled.")
|
| 499 |
elif roof_mask_resized is None:
|
|
|
|
| 527 |
water_mask_pct=mask_pct(water_mask_resized) if request.use_water_mask else None,
|
| 528 |
road_mask_pct=mask_pct(road_mask_resized) if request.use_road_mask else None,
|
| 529 |
roof_mask_pct=mask_pct(roof_mask_resized) if request.use_roof_mask else None,
|
| 530 |
+
tree_mask_pct=mask_pct(tree_mask_resized) if request.use_tree_mask else None,
|
| 531 |
water_mask_enabled=request.use_water_mask,
|
| 532 |
road_mask_enabled=request.use_road_mask,
|
| 533 |
roof_mask_enabled=request.use_roof_mask,
|
| 534 |
+
tree_mask_enabled=request.use_tree_mask,
|
| 535 |
used_valid_center=used_valid_center,
|
| 536 |
warnings=warnings,
|
| 537 |
std_thresh_applied=std_thresh_eff,
|
app/segmentation.py
CHANGED
|
@@ -1,6 +1,7 @@
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
| 3 |
from dataclasses import dataclass
|
|
|
|
| 4 |
import re
|
| 5 |
|
| 6 |
import numpy as np
|
|
@@ -9,6 +10,7 @@ from PIL import Image
|
|
| 9 |
|
| 10 |
from .config import (
|
| 11 |
ROAD_PROMPT,
|
|
|
|
| 12 |
SEGMENTATION_MASK_THRESH,
|
| 13 |
SEGMENTATION_MAX_SIDE,
|
| 14 |
SEGMENTATION_MODEL_ID,
|
|
@@ -120,10 +122,12 @@ class SegmenterRequest:
|
|
| 120 |
source_path: Optional[str] = None
|
| 121 |
want_water: bool = False
|
| 122 |
want_road: bool = False
|
|
|
|
| 123 |
want_tree: bool = False
|
| 124 |
max_side: int = SEGMENTATION_MAX_SIDE
|
| 125 |
water_prompt: str = WATER_PROMPT
|
| 126 |
road_prompt: str = ROAD_PROMPT
|
|
|
|
| 127 |
tree_prompt: str = TREE_PROMPT
|
| 128 |
score_threshold: float = SEGMENTATION_SCORE_THRESH
|
| 129 |
mask_threshold: float = SEGMENTATION_MASK_THRESH
|
|
@@ -147,7 +151,7 @@ class SegmenterService:
|
|
| 147 |
return self._segmenters[model_id]
|
| 148 |
|
| 149 |
def get_masks(self, request: SegmenterRequest, model_id: str | None = None) -> dict[str, np.ndarray]:
|
| 150 |
-
if not (request.want_water or request.want_road or request.want_tree):
|
| 151 |
return {}
|
| 152 |
segmenter = self._get_segmenter(model_id or self.model_id)
|
| 153 |
prompts: dict[str, str] = {}
|
|
@@ -155,6 +159,8 @@ class SegmenterService:
|
|
| 155 |
prompts["water"] = request.water_prompt
|
| 156 |
if request.want_road and request.road_prompt:
|
| 157 |
prompts["road"] = request.road_prompt
|
|
|
|
|
|
|
| 158 |
if request.want_tree and request.tree_prompt:
|
| 159 |
prompts["tree"] = request.tree_prompt
|
| 160 |
try:
|
|
@@ -173,6 +179,8 @@ class SegmenterService:
|
|
| 173 |
result["water"] = masks["water"]
|
| 174 |
if request.want_road and masks.get("road") is not None:
|
| 175 |
result["road"] = masks["road"]
|
|
|
|
|
|
|
| 176 |
if request.want_tree and masks.get("tree") is not None:
|
| 177 |
result["tree"] = masks["tree"]
|
| 178 |
return result
|
|
|
|
| 1 |
from __future__ import annotations
|
| 2 |
|
| 3 |
from dataclasses import dataclass
|
| 4 |
+
from typing import Dict, Optional
|
| 5 |
import re
|
| 6 |
|
| 7 |
import numpy as np
|
|
|
|
| 10 |
|
| 11 |
from .config import (
|
| 12 |
ROAD_PROMPT,
|
| 13 |
+
ROOF_PROMPT,
|
| 14 |
SEGMENTATION_MASK_THRESH,
|
| 15 |
SEGMENTATION_MAX_SIDE,
|
| 16 |
SEGMENTATION_MODEL_ID,
|
|
|
|
| 122 |
source_path: Optional[str] = None
|
| 123 |
want_water: bool = False
|
| 124 |
want_road: bool = False
|
| 125 |
+
want_roof: bool = False
|
| 126 |
want_tree: bool = False
|
| 127 |
max_side: int = SEGMENTATION_MAX_SIDE
|
| 128 |
water_prompt: str = WATER_PROMPT
|
| 129 |
road_prompt: str = ROAD_PROMPT
|
| 130 |
+
roof_prompt: str = ROOF_PROMPT
|
| 131 |
tree_prompt: str = TREE_PROMPT
|
| 132 |
score_threshold: float = SEGMENTATION_SCORE_THRESH
|
| 133 |
mask_threshold: float = SEGMENTATION_MASK_THRESH
|
|
|
|
| 151 |
return self._segmenters[model_id]
|
| 152 |
|
| 153 |
def get_masks(self, request: SegmenterRequest, model_id: str | None = None) -> dict[str, np.ndarray]:
|
| 154 |
+
if not (request.want_water or request.want_road or request.want_tree or request.want_roof):
|
| 155 |
return {}
|
| 156 |
segmenter = self._get_segmenter(model_id or self.model_id)
|
| 157 |
prompts: dict[str, str] = {}
|
|
|
|
| 159 |
prompts["water"] = request.water_prompt
|
| 160 |
if request.want_road and request.road_prompt:
|
| 161 |
prompts["road"] = request.road_prompt
|
| 162 |
+
if request.want_roof and request.roof_prompt:
|
| 163 |
+
prompts["roof"] = request.roof_prompt
|
| 164 |
if request.want_tree and request.tree_prompt:
|
| 165 |
prompts["tree"] = request.tree_prompt
|
| 166 |
try:
|
|
|
|
| 179 |
result["water"] = masks["water"]
|
| 180 |
if request.want_road and masks.get("road") is not None:
|
| 181 |
result["road"] = masks["road"]
|
| 182 |
+
if request.want_roof and masks.get("roof") is not None:
|
| 183 |
+
result["roof"] = masks["roof"]
|
| 184 |
if request.want_tree and masks.get("tree") is not None:
|
| 185 |
result["tree"] = masks["tree"]
|
| 186 |
return result
|
app/ui.py
CHANGED
|
@@ -23,6 +23,7 @@ def _make_request(
|
|
| 23 |
use_tree_mask,
|
| 24 |
water_prompt,
|
| 25 |
road_prompt,
|
|
|
|
| 26 |
tree_prompt,
|
| 27 |
altitude_m,
|
| 28 |
fov_deg,
|
|
@@ -47,6 +48,7 @@ def _make_request(
|
|
| 47 |
use_tree_mask=use_tree_mask,
|
| 48 |
water_prompt=water_prompt,
|
| 49 |
road_prompt=road_prompt,
|
|
|
|
| 50 |
tree_prompt=tree_prompt,
|
| 51 |
altitude_m=altitude_m,
|
| 52 |
fov_deg=fov_deg,
|
|
@@ -68,11 +70,12 @@ def _format_status(summary: AnalysisSummary | None) -> str:
|
|
| 68 |
if not summary:
|
| 69 |
return "**Status**\nAwaiting analysis."
|
| 70 |
masks_line = " / ".join(
|
| 71 |
-
f"{label}:{'
|
| 72 |
-
for label, enabled
|
| 73 |
-
("Water", summary.water_mask_enabled
|
| 74 |
-
("Road", summary.road_mask_enabled
|
| 75 |
-
("
|
|
|
|
| 76 |
)
|
| 77 |
)
|
| 78 |
warning_text = ""
|
|
@@ -83,8 +86,8 @@ def _format_status(summary: AnalysisSummary | None) -> str:
|
|
| 83 |
f"- Model: `{summary.model_id}`\n"
|
| 84 |
f"- Process res: {summary.process_resolution}px; Runtime: {summary.runtime_ms:.0f} ms\n"
|
| 85 |
f"- Footprint: {summary.footprint_m:.1f} m (~{summary.footprint_image_px}px)\n"
|
| 86 |
-
f"- Safe: {summary.safe_area_pct:.1f}% | Hazard: {summary.hazard_pct:.1f}
|
| 87 |
-
f"- Masks: {masks_line}"
|
| 88 |
f"{warning_text}"
|
| 89 |
)
|
| 90 |
|
|
@@ -179,12 +182,14 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 179 |
step=0.01,
|
| 180 |
info="Lower suppresses slopes/edges; higher tolerates tilt.",
|
| 181 |
)
|
| 182 |
-
with gr.
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
|
|
|
|
|
|
| 188 |
with gr.Accordion("Segmentation settings", open=False):
|
| 189 |
gr.Markdown("Control SAM3 and prompts.")
|
| 190 |
segmentation_model_id = gr.Dropdown(
|
|
@@ -213,6 +218,11 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 213 |
value=defaults.road_prompt,
|
| 214 |
placeholder="e.g., road",
|
| 215 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
tree_prompt = gr.Textbox(
|
| 217 |
label="Tree prompt",
|
| 218 |
value=defaults.tree_prompt,
|
|
@@ -288,14 +298,14 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 288 |
with gr.Column(scale=2, min_width=520, elem_id="preview-wrap", elem_classes="preview-col"):
|
| 289 |
main_view = gr.Image(
|
| 290 |
label="Analyzed",
|
| 291 |
-
height=
|
| 292 |
elem_id="main-preview",
|
| 293 |
show_download_button=True,
|
| 294 |
show_fullscreen_button=False,
|
| 295 |
)
|
| 296 |
orig_view = gr.Image(
|
| 297 |
label="Original",
|
| 298 |
-
height=
|
| 299 |
show_download_button=True,
|
| 300 |
show_fullscreen_button=False,
|
| 301 |
)
|
|
@@ -346,6 +356,7 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 346 |
use_tree_mask,
|
| 347 |
water_prompt,
|
| 348 |
road_prompt,
|
|
|
|
| 349 |
tree_prompt,
|
| 350 |
altitude_m,
|
| 351 |
fov_deg,
|
|
@@ -381,6 +392,7 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 381 |
use_tree_mask,
|
| 382 |
water_prompt,
|
| 383 |
road_prompt,
|
|
|
|
| 384 |
tree_prompt,
|
| 385 |
altitude_m,
|
| 386 |
fov_deg,
|
|
@@ -432,6 +444,7 @@ def build_ui(analyzer: SafetyAnalyzer | None = None) -> gr.Blocks:
|
|
| 432 |
use_tree_mask,
|
| 433 |
water_prompt,
|
| 434 |
road_prompt,
|
|
|
|
| 435 |
tree_prompt,
|
| 436 |
altitude_m,
|
| 437 |
fov_deg,
|
|
|
|
| 23 |
use_tree_mask,
|
| 24 |
water_prompt,
|
| 25 |
road_prompt,
|
| 26 |
+
roof_prompt,
|
| 27 |
tree_prompt,
|
| 28 |
altitude_m,
|
| 29 |
fov_deg,
|
|
|
|
| 48 |
use_tree_mask=use_tree_mask,
|
| 49 |
water_prompt=water_prompt,
|
| 50 |
road_prompt=road_prompt,
|
| 51 |
+
roof_prompt=roof_prompt,
|
| 52 |
tree_prompt=tree_prompt,
|
| 53 |
altitude_m=altitude_m,
|
| 54 |
fov_deg=fov_deg,
|
|
|
|
| 70 |
if not summary:
|
| 71 |
return "**Status**\nAwaiting analysis."
|
| 72 |
masks_line = " / ".join(
|
| 73 |
+
f"{label}:{'on' if enabled else 'off'}"
|
| 74 |
+
for label, enabled in (
|
| 75 |
+
("Water", summary.water_mask_enabled),
|
| 76 |
+
("Road", summary.road_mask_enabled),
|
| 77 |
+
("Tree", summary.tree_mask_enabled),
|
| 78 |
+
("Roof", summary.roof_mask_enabled),
|
| 79 |
)
|
| 80 |
)
|
| 81 |
warning_text = ""
|
|
|
|
| 86 |
f"- Model: `{summary.model_id}`\n"
|
| 87 |
f"- Process res: {summary.process_resolution}px; Runtime: {summary.runtime_ms:.0f} ms\n"
|
| 88 |
f"- Footprint: {summary.footprint_m:.1f} m (~{summary.footprint_image_px}px)\n"
|
| 89 |
+
f"- Safe: {summary.safe_area_pct:.1f}% | Hazard: {summary.hazard_pct:.1f}%"
|
| 90 |
+
f"\n- Masks: {masks_line}"
|
| 91 |
f"{warning_text}"
|
| 92 |
)
|
| 93 |
|
|
|
|
| 182 |
step=0.01,
|
| 183 |
info="Lower suppresses slopes/edges; higher tolerates tilt.",
|
| 184 |
)
|
| 185 |
+
with gr.Accordion("Hazard segmentation", open=False):
|
| 186 |
+
gr.Markdown("Select which hazards to mask using SAM3 segmentation.")
|
| 187 |
+
with gr.Row():
|
| 188 |
+
use_water_mask = gr.Checkbox(label="Exclude water", value=True, info="Mask water regions.")
|
| 189 |
+
use_road_mask = gr.Checkbox(label="Exclude roads", value=True, info="Mask road surfaces.")
|
| 190 |
+
with gr.Row():
|
| 191 |
+
use_tree_mask = gr.Checkbox(label="Exclude trees", value=True, info="Mask trees/foliage.")
|
| 192 |
+
use_roof_mask = gr.Checkbox(label="Exclude rooftops", value=True, info="Mask rooftop areas.")
|
| 193 |
with gr.Accordion("Segmentation settings", open=False):
|
| 194 |
gr.Markdown("Control SAM3 and prompts.")
|
| 195 |
segmentation_model_id = gr.Dropdown(
|
|
|
|
| 218 |
value=defaults.road_prompt,
|
| 219 |
placeholder="e.g., road",
|
| 220 |
)
|
| 221 |
+
roof_prompt = gr.Textbox(
|
| 222 |
+
label="Roof prompt",
|
| 223 |
+
value=defaults.roof_prompt,
|
| 224 |
+
placeholder="e.g., roof",
|
| 225 |
+
)
|
| 226 |
tree_prompt = gr.Textbox(
|
| 227 |
label="Tree prompt",
|
| 228 |
value=defaults.tree_prompt,
|
|
|
|
| 298 |
with gr.Column(scale=2, min_width=520, elem_id="preview-wrap", elem_classes="preview-col"):
|
| 299 |
main_view = gr.Image(
|
| 300 |
label="Analyzed",
|
| 301 |
+
height=485,
|
| 302 |
elem_id="main-preview",
|
| 303 |
show_download_button=True,
|
| 304 |
show_fullscreen_button=False,
|
| 305 |
)
|
| 306 |
orig_view = gr.Image(
|
| 307 |
label="Original",
|
| 308 |
+
height=485,
|
| 309 |
show_download_button=True,
|
| 310 |
show_fullscreen_button=False,
|
| 311 |
)
|
|
|
|
| 356 |
use_tree_mask,
|
| 357 |
water_prompt,
|
| 358 |
road_prompt,
|
| 359 |
+
roof_prompt,
|
| 360 |
tree_prompt,
|
| 361 |
altitude_m,
|
| 362 |
fov_deg,
|
|
|
|
| 392 |
use_tree_mask,
|
| 393 |
water_prompt,
|
| 394 |
road_prompt,
|
| 395 |
+
roof_prompt,
|
| 396 |
tree_prompt,
|
| 397 |
altitude_m,
|
| 398 |
fov_deg,
|
|
|
|
| 444 |
use_tree_mask,
|
| 445 |
water_prompt,
|
| 446 |
road_prompt,
|
| 447 |
+
roof_prompt,
|
| 448 |
tree_prompt,
|
| 449 |
altitude_m,
|
| 450 |
fov_deg,
|
demo/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
# Makes the curated demo importable as a package (demo.*).
|
demo/curated.py
CHANGED
|
@@ -29,38 +29,35 @@ class CuratedSample:
|
|
| 29 |
|
| 30 |
def format_status(summary: Optional[Dict[str, Any]]) -> str:
|
| 31 |
if not summary:
|
| 32 |
-
return "**Status
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
f"
|
| 49 |
-
f"
|
| 50 |
-
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
|
| 54 |
def format_metrics(summary: Optional[Dict[str, Any]]) -> str:
|
| 55 |
if not summary:
|
| 56 |
return "No metrics yet. Run the analyzer to populate this section."
|
| 57 |
-
lines = [
|
| 58 |
-
f"**Safe coverage:** {summary.get('safe_area_pct', 0.0):.1f}% of frame",
|
| 59 |
-
f"**Hazard coverage:** {summary.get('hazard_pct', 0.0):.1f}%",
|
| 60 |
-
f"**Landing center (px):** {summary.get('landing_center_image', ['-', '-'])[0]}, {summary.get('landing_center_image', ['-', '-'])[1]}",
|
| 61 |
-
f"**Footprint size:** {summary.get('footprint_m', 0.0):.1f} m ≈ {summary.get('footprint_image_px', 0)}px",
|
| 62 |
-
f"**Effective thresholds:** std ≤ {summary.get('std_thresh_applied', 0.0):.4f}, grad ≤ {summary.get('grad_thresh_applied', 0.0):.3f}",
|
| 63 |
-
]
|
| 64 |
if not summary.get("used_valid_center", True):
|
| 65 |
lines.append("Warning: No fully safe footprint; showing lowest-variance patch.")
|
| 66 |
warnings = summary.get("warnings") or []
|
|
|
|
| 29 |
|
| 30 |
def format_status(summary: Optional[Dict[str, Any]]) -> str:
|
| 31 |
if not summary:
|
| 32 |
+
return "**Status**\nAwaiting analysis."
|
| 33 |
+
masks_line = " / ".join(
|
| 34 |
+
f"{label}:{'on' if summary.get(enabled_key) else 'off'}"
|
| 35 |
+
for label, enabled_key in (
|
| 36 |
+
("Water", "water_mask_enabled"),
|
| 37 |
+
("Road", "road_mask_enabled"),
|
| 38 |
+
("Tree", "tree_mask_enabled"),
|
| 39 |
+
("Roof", "roof_mask_enabled"),
|
| 40 |
+
)
|
| 41 |
+
)
|
| 42 |
+
warnings = summary.get("warnings") or []
|
| 43 |
+
warning_text = ""
|
| 44 |
+
if warnings:
|
| 45 |
+
warning_text = "\nWarnings: " + " | ".join(warnings)
|
| 46 |
+
return (
|
| 47 |
+
"**Run Status**\n"
|
| 48 |
+
f"- Model: `{summary.get('model_id')}`\n"
|
| 49 |
+
f"- Process res: {summary.get('process_resolution')}px; Runtime: {summary.get('runtime_ms', 0):.0f} ms\n"
|
| 50 |
+
f"- Footprint: {summary.get('footprint_m', 0.0):.1f} m (~{summary.get('footprint_image_px', 0)}px)\n"
|
| 51 |
+
f"- Safe: {summary.get('safe_area_pct', 0.0):.1f}% | Hazard: {summary.get('hazard_pct', 0.0):.1f}%\n"
|
| 52 |
+
f"- Masks: {masks_line}"
|
| 53 |
+
f"{warning_text}"
|
| 54 |
+
)
|
| 55 |
|
| 56 |
|
| 57 |
def format_metrics(summary: Optional[Dict[str, Any]]) -> str:
|
| 58 |
if not summary:
|
| 59 |
return "No metrics yet. Run the analyzer to populate this section."
|
| 60 |
+
lines: list[str] = []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
if not summary.get("used_valid_center", True):
|
| 62 |
lines.append("Warning: No fully safe footprint; showing lowest-variance patch.")
|
| 63 |
warnings = summary.get("warnings") or []
|
demo/curated_ui.py
CHANGED
|
@@ -59,13 +59,10 @@ def build_curated_ui(index_path: str | Path | None = None) -> gr.Blocks:
|
|
| 59 |
"DepthAnything3 + segmentation + landing-safety scoring, showcased on VISLOC scenes. "
|
| 60 |
"Each sample pairs the raw RGB with a safety overlay that highlights flat, obstacle-free landing zones."
|
| 61 |
)
|
| 62 |
-
gr.Markdown(
|
| 63 |
-
"1) Pick a sample below. 2) Compare the RGB and safety overlay. 3) Scan the status/metrics to see why that spot was chosen."
|
| 64 |
-
)
|
| 65 |
gr.HTML(
|
| 66 |
"""
|
| 67 |
<div style="background: #0d1117; color: #e6edf3; padding: 10px 12px; border-radius: 10px; font-size: 13px;">
|
| 68 |
-
<strong>Legend</strong>: safe mask = <span style="color:#00ff00;">green</span
|
| 69 |
</div>
|
| 70 |
"""
|
| 71 |
)
|
|
@@ -94,10 +91,10 @@ def build_curated_ui(index_path: str | Path | None = None) -> gr.Blocks:
|
|
| 94 |
gr.Markdown("**Sample overview**")
|
| 95 |
description_card = gr.Markdown()
|
| 96 |
with gr.Column():
|
| 97 |
-
gr.Markdown("**Status
|
| 98 |
status_card = gr.Markdown()
|
| 99 |
with gr.Column():
|
| 100 |
-
gr.Markdown("**
|
| 101 |
metrics_card = gr.Markdown()
|
| 102 |
|
| 103 |
demo.load(
|
|
|
|
| 59 |
"DepthAnything3 + segmentation + landing-safety scoring, showcased on VISLOC scenes. "
|
| 60 |
"Each sample pairs the raw RGB with a safety overlay that highlights flat, obstacle-free landing zones."
|
| 61 |
)
|
|
|
|
|
|
|
|
|
|
| 62 |
gr.HTML(
|
| 63 |
"""
|
| 64 |
<div style="background: #0d1117; color: #e6edf3; padding: 10px 12px; border-radius: 10px; font-size: 13px;">
|
| 65 |
+
<strong>Legend</strong>: safe mask = <span style="color:#00ff00;">green fill + outline</span>; depth hazards = <span style="color:#ff4d4f;">red</span> heat; segmentation hazards (water/road/tree/roof) = <span style="color:#e6edf3;">black, hatched overlays</span>; landing spot = <span style="color:#ff8c00;">orange box + crosshair</span>.
|
| 66 |
</div>
|
| 67 |
"""
|
| 68 |
)
|
|
|
|
| 91 |
gr.Markdown("**Sample overview**")
|
| 92 |
description_card = gr.Markdown()
|
| 93 |
with gr.Column():
|
| 94 |
+
gr.Markdown("**Status**")
|
| 95 |
status_card = gr.Markdown()
|
| 96 |
with gr.Column():
|
| 97 |
+
gr.Markdown("**Metrics**")
|
| 98 |
metrics_card = gr.Markdown()
|
| 99 |
|
| 100 |
demo.load(
|
demo/demo_app.py
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Launch the curated (precomputed) Landing Site Safety Gradio demo."""
|
| 3 |
+
|
| 4 |
+
from __future__ import annotations
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
import sys
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
# Ensure repository root on sys.path so `demo.*` imports work when run as a script.
|
| 11 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 12 |
+
if str(ROOT) not in sys.path:
|
| 13 |
+
sys.path.insert(0, str(ROOT))
|
| 14 |
+
|
| 15 |
+
from demo.curated_ui import build_curated_ui
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def main() -> None:
|
| 19 |
+
index_override = os.getenv("CURATED_INDEX_PATH")
|
| 20 |
+
demo = build_curated_ui(index_override)
|
| 21 |
+
|
| 22 |
+
use_queue = os.getenv("DA_USE_QUEUE")
|
| 23 |
+
use_queue_flag = False if use_queue is None else use_queue.lower() not in {"0", "false", "no"}
|
| 24 |
+
share = os.getenv("DA_SHARE")
|
| 25 |
+
share_flag = False if share is None else share.lower() not in {"0", "false", "no"}
|
| 26 |
+
server_port_str = os.getenv("GRADIO_SERVER_PORT")
|
| 27 |
+
server_port = int(server_port_str) if server_port_str else None
|
| 28 |
+
server_port_range = None
|
| 29 |
+
range_env = os.getenv("GRADIO_SERVER_PORT_RANGE")
|
| 30 |
+
if range_env:
|
| 31 |
+
try:
|
| 32 |
+
start_str, end_str = range_env.split(",", 1)
|
| 33 |
+
server_port_range = (int(start_str), int(end_str))
|
| 34 |
+
except ValueError:
|
| 35 |
+
server_port_range = None
|
| 36 |
+
launch_kwargs = {"share": share_flag}
|
| 37 |
+
if server_port is not None:
|
| 38 |
+
launch_kwargs["server_port"] = server_port
|
| 39 |
+
if server_port_range is not None:
|
| 40 |
+
launch_kwargs["server_port_range"] = server_port_range
|
| 41 |
+
if use_queue_flag:
|
| 42 |
+
try:
|
| 43 |
+
demo.queue().launch(**launch_kwargs)
|
| 44 |
+
except TypeError:
|
| 45 |
+
launch_kwargs.pop("server_port_range", None)
|
| 46 |
+
demo.queue().launch(**launch_kwargs)
|
| 47 |
+
else:
|
| 48 |
+
try:
|
| 49 |
+
demo.launch(**launch_kwargs)
|
| 50 |
+
except TypeError:
|
| 51 |
+
launch_kwargs.pop("server_port_range", None)
|
| 52 |
+
demo.launch(**launch_kwargs)
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
if __name__ == "__main__":
|
| 56 |
+
main()
|
demo/precompute_curated.py
CHANGED
|
@@ -70,6 +70,7 @@ def _analysis_request_from_args(args: argparse.Namespace, source_path: Path) ->
|
|
| 70 |
use_tree_mask=True,
|
| 71 |
water_prompt=resolve(args.water_prompt, defaults.water_prompt),
|
| 72 |
road_prompt=resolve(args.road_prompt, defaults.road_prompt),
|
|
|
|
| 73 |
tree_prompt=resolve(getattr(args, "tree_prompt", None), defaults.tree_prompt),
|
| 74 |
altitude_m=float(resolve(args.altitude_m, defaults.altitude_m)),
|
| 75 |
fov_deg=float(resolve(args.fov_deg, defaults.fov_deg)),
|
|
@@ -151,11 +152,9 @@ def precompute_curated(
|
|
| 151 |
base_view=base_view,
|
| 152 |
heat_on=True,
|
| 153 |
heat_alpha=float(heat_opacity),
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
road_on=True,
|
| 158 |
-
tree_on=True,
|
| 159 |
grad_on=False,
|
| 160 |
flat_on=False,
|
| 161 |
flat_heat_on=False,
|
|
@@ -237,6 +236,7 @@ def build_parser() -> argparse.ArgumentParser:
|
|
| 237 |
p.add_argument("--fov-deg", type=float, help="Camera FOV in degrees.")
|
| 238 |
p.add_argument("--water-prompt", type=str, help="Water segmentation prompt.")
|
| 239 |
p.add_argument("--road-prompt", type=str, help="Road segmentation prompt.")
|
|
|
|
| 240 |
p.add_argument("--tree-prompt", type=str, help="Tree segmentation prompt.")
|
| 241 |
p.add_argument("--use-water-mask", action="store_true", dest="use_water_mask", help="Enable water mask.")
|
| 242 |
p.add_argument("--no-water-mask", action="store_false", dest="use_water_mask", help="Disable water mask.")
|
|
|
|
| 70 |
use_tree_mask=True,
|
| 71 |
water_prompt=resolve(args.water_prompt, defaults.water_prompt),
|
| 72 |
road_prompt=resolve(args.road_prompt, defaults.road_prompt),
|
| 73 |
+
roof_prompt=resolve(getattr(args, "roof_prompt", None), defaults.roof_prompt),
|
| 74 |
tree_prompt=resolve(getattr(args, "tree_prompt", None), defaults.tree_prompt),
|
| 75 |
altitude_m=float(resolve(args.altitude_m, defaults.altitude_m)),
|
| 76 |
fov_deg=float(resolve(args.fov_deg, defaults.fov_deg)),
|
|
|
|
| 152 |
base_view=base_view,
|
| 153 |
heat_on=True,
|
| 154 |
heat_alpha=float(heat_opacity),
|
| 155 |
+
risk_on=True,
|
| 156 |
+
risk_alpha=float(hazard_opacity),
|
| 157 |
+
hazards_on=True,
|
|
|
|
|
|
|
| 158 |
grad_on=False,
|
| 159 |
flat_on=False,
|
| 160 |
flat_heat_on=False,
|
|
|
|
| 236 |
p.add_argument("--fov-deg", type=float, help="Camera FOV in degrees.")
|
| 237 |
p.add_argument("--water-prompt", type=str, help="Water segmentation prompt.")
|
| 238 |
p.add_argument("--road-prompt", type=str, help="Road segmentation prompt.")
|
| 239 |
+
p.add_argument("--roof-prompt", type=str, help="Roof segmentation prompt.")
|
| 240 |
p.add_argument("--tree-prompt", type=str, help="Tree segmentation prompt.")
|
| 241 |
p.add_argument("--use-water-mask", action="store_true", dest="use_water_mask", help="Enable water mask.")
|
| 242 |
p.add_argument("--no-water-mask", action="store_false", dest="use_water_mask", help="Disable water mask.")
|