ARCHITECTURE.md

# Landing Site Safety Analyzer – Architecture and Calculations

This document describes the flow in the current Gradio app (`app/ui.py`), from input selection through model inference, safety scoring, and UI composition.

## Data and Models
- **Inputs**: Images under `data/Image/` (VISLOC and any custom folders) via `list_all_data_inputs`, with a 5% border crop (`crop_nonblack`) to drop black padding. Supported extensions: jpg/jpeg/png (any case).
- **Depth model**: Depth Anything 3, cached per model id (`DepthEngine`). Inference caps the long side to `process_res_cap` (default 1024) using `upper_bound_resize` before predicting.
- **Segmentation model**: SAM3 (`facebook/sam3`) for promptable water/road/tree/roof masking. The segmenter is cached per model id but masks are recomputed every run (no output cache). Default `segmentation_max_side` is 512 and is clamped to the depth resolution (min 128).

## Constants and Defaults
- Altitude/FOV defaults: 450 m, 90° (footprint default 10 m).
- Thresholds: `std_thresh` default 0.005, `grad_thresh` default 0.1; both auto-scale with depth resolution so sliders act as base values.
- Clearance factor: default 1.0 (dilates hazards by the footprint size).
- Coverage strictness: default 0.95 (fraction of the footprint that must be safe).
- Texture threshold: default 0.3 (suppresses highly textured regions).
- Depth smoothing is supported but set to 0.0 in the UI (effectively off).
- Roof mask: SAM3 promptable segmentation (default prompt: `roof`), resized to depth scale and expanded to footprint size; no depth-based roof heuristics remain.

## Per-Image Processing Pipeline
1. **Load and crop** the selected image (RGB, 5% border removed).
2. **Depth inference**: Run DA3 with long side clamped to `process_res_cap`; obtain `depth_raw`, then detrend with `remove_global_plane`. Optional Gaussian blur uses `depth_smoothing_base * res_scale` (currently zero).
3. **Footprint sizing**:
   - `fx = (W/2) / tan(FOV/2)` where `W` is depth width.
   - `patch_px = footprint_m * fx / altitude_m`, clamped to bounds and forced odd; `half_span = patch_px//2`.
   - Visualization window `vis_patch` is an odd size capped to 1/8 of the smallest depth dimension for sharper std previews.
4. **Texture mask**: Sobel magnitude on the RGB (blurred by `patch_px/40`), normalized; pixels above `texture_threshold` are suppressed.
5. **Segmentation masks (optional)**:
   - Water/Road/Tree/Roof via SAM3 at `segmentation_max_side`, with text prompts. Instance masks are unioned per class, resized to depth scale, and dilated to footprint size for blocking.
6. **Flat region search (`pick_flat_patch`)**:
   - Normalize depth to [0,1], compute `std_map` via box mean/mean_sq, and `grad_norm` via `np.gradient` normalized at the 95th percentile.
   - Landing mask starts from `grad_norm < grad_thresh_eff`, excludes water if present, and keeps the lowest-variance patch as a fallback box.
7. **Safe mask construction**:
   - Base safe mask: `(std_map < std_thresh_eff) & (grad_norm < grad_thresh_eff) & landing_mask & texture_mask`.
   - Apply segmentation blocks (expanded masks) to remove water/road/tree/roof regions.
   - Clearance: dilate hazards by `clearance_factor * patch_px` (default 1.0).
   - Coverage: box filter with `patch_px` window; keep pixels meeting `coverage_strictness` (default 0.95).
   - Drop small components (< footprint area).
8. **Center selection**:
   - Prefer centers where full-footprint coverage exists; choose the largest component and rank by distance transform minus flatness penalty (`openness_weight`).
   - If no full coverage but safe pixels exist, pick the safest point inside the safe mask (distance vs. flatness).
   - Fallbacks: landing mask with segmentation removed; if empty, use the flattest patch center.
   - Convert depth center to image space; footprint box is scaled to image pixels and clamped to a minimum of 3 px.
9. **Visualization layers**:
   - Depth colormap from `depth_raw`.
   - Flatness std preview (`std_map_vis`), gradient magnitude, gradient mask, flatness heatmap overlay.
   - Water/Road/Tree/Roof masks and per-class hazard overlays.
   - Safety overlays: green safe heatmap, red hazard overlay from `risk_map`, grayscale safety score, landing spot box/crosshair.

## Safety Heatmap and Hazards
- `safe_mask` drives the green overlay (alpha per pixel).
- Hazard overlay uses `risk_map` (max of std/grad over-threshold). Pixels above `risk_threshold` are emphasized; water/road/tree hazards can also be overlaid separately.

## Overlay Composition (`compose_view`)
- Base view: one of the named layers (RGB/Depth/Flatness/Gradient/Gradient mask/Water mask/Road mask/Tree mask/Safety score/Safety heatmap overlay).
- Overlays: safety heatmap, hazard heatmap, per-class hazards (water/road/tree), gradient, optional landing spot box. Fixed alpha values; toggling overlays does not rerun inference.
- Returned image is RGB.

## Caching and State
- Depth model cache keyed by model id (`DepthEngine`); default model is preloaded.
- SAM3 models are cached per id; masks are not cached and are recomputed every run to reflect real-time cost.
- `images_state` holds the latest rendered layers; overlay-only changes don’t rerun inference. Prompt changes only re-trigger processing on submit/Run, not every keystroke.

## User Controls and Effects
- `process_res_cap`: depth max side (px) for DA3.
- `footprint_m`, `altitude_m`, `fov_deg`: determine footprint size in pixels.
- `std_thresh`, `grad_thresh`, `texture_threshold`: safety criteria.
- `clearance_factor`: hazard dilation (default 1.0).
- `coverage_strictness`, `openness_weight`: coverage tolerance and center ranking.
- Segmentation toggles, prompts, `segmentation_max_side`, `segmentation_score_thresh`, `segmentation_mask_thresh`: control SAM3 masks for water/road/tree.
- Overlay toggles affect only display; inference results are reused.

## Error Handling
- Bad/missing inputs raise Gradio errors.
- Segmentation failures warn and proceed without that mask; water/road/tree/roof warnings clarify when masks are disabled or not detected.
- Coverage/boxFilter fallbacks keep processing even if OpenCV operations fail.

## Outputs
- Dict of PIL Images keyed by: RGB, Depth, Flatness map (std), Depth gradient, Gradient mask, Water mask, Road mask, Tree mask, Roof mask, Safety heatmap overlay, Hazard overlay, Water/Road/Tree hazard overlays, Flatness heatmap overlay, Safety score (grayscale), Landing spot overlay.
- Run summaries surface model id, process resolution, runtime, footprint size (depth + image scale), landing center, safe/hazard coverage, effective thresholds, per-mask coverage (water/road/tree/roof), and warnings for disabled/absent masks or missing safe regions; the UI cards render these fields directly.
- `compose_view` uses these to build the preview.