Add confidence threshold control

Co-authored-by: OpenAI Codex <codex@openai.com>

AGENTS.md

@@ -261,6 +261,14 @@ technical cutaway animation.
   reveal. Local browser verification confirmed the scene still renders, orbit
   drag still works, the canvas remains the pointer target, the source preview
   remains contained, and the mobile layout has no horizontal overflow.
+- Implemented the `FEATURE.md` confidence-threshold control on June 14, 2026:
+  added a toolbar slider defaulting to `0.5`, made the browser recompute render
+  mode from cached analysis plus the current slider value, and kept slider
+  changes local with no extra `/analyze_image` or `/generate_scene` calls.
+  Added `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` for the server's advisory fallback.
+  Local verification passed for schema/parser checks, FastAPI `TestClient`,
+  slider downgrade/promote behavior, annotated-photo fallback, keyboard slider
+  operation, mobile no-overflow layout, and canvas pointer targeting.
 
 ## Next Work
 

FEATURE.md

@@ -0,0 +1,194 @@
+# FEATURE: Confidence Threshold Control for Cutaway Generation
+
+Status: Implemented locally by Codex, June 14, 2026.
+Author of spec: codebase review pass, June 14, 2026.
+
+## Summary
+
+Add a user-facing **confidence threshold slider** that controls how high the
+model's analysis `confidence` must be before the deterministic Three.js cutaway
+is rendered. Today this threshold exists but is a hardcoded `0.5` magic number,
+duplicated across Python and JavaScript, and not user-controllable.
+
+## Findings (current behavior)
+
+**Q: Does the generation of the Three.js parts have a confidence threshold to
+pass before sending back to the user?**
+
+Partially yes, but it is implicit, hardcoded, and not a true block:
+
+1. The vision model returns a `confidence` value (`0`–`1`) in the analysis JSON.
+   See `snap2sim/schema.py` (`ANALYSIS_SCHEMA`, `confidence` is
+   `number, minimum 0, maximum 1`), `snap2sim/prompts.py`, and
+   `snap2sim/model_io.py` (defaults: coerced `0.55`, generic fallback `0.45`).
+
+2. Confidence gates the **render mode**, not whether output is returned. The
+   threshold is a hardcoded `0.5` and lives in **two** places:
+   - **Python** `snap2sim/schema.py:218` `select_render_mode()`:
+     `low_confidence = _is_number(confidence) and float(confidence) < 0.5`.
+     This is called by `snap2sim/backend.py:47` `InferenceClient.generate_scene()`,
+     which returns `render_mode` in the `/generate_scene` payload.
+   - **JavaScript** `index.html:760` `chooseRenderMode()`:
+     `confidence < 0.5` / `confidence >= 0.5`. Used as a client-side fallback
+     only when the server payload omits `render_mode`.
+
+3. The threshold **downgrades** rather than blocks. The chain is:
+   `three` (3D cutaway) -> `annotate` (annotated source photo overlay) ->
+   `unavailable` (honest "cannot render" state). A low-confidence analysis is
+   still returned to the user, just not as 3D parts. See
+   `index.html:748` `renderScenePayload()` and `index.html:760`
+   `chooseRenderMode()`; mirrored in `schema.py:218` `select_render_mode()`.
+
+4. Confidence is currently only **displayed** (as a percentage) at
+   `index.html:726`; there is no control to change the cutoff.
+
+5. There is **no server-side hard gate**. `modal_app.py` / `app.py` validate and
+   coerce the payload but never reject on low confidence.
+
+**Conclusion:** an implicit confidence threshold (`0.5`) already governs whether
+Three.js parts render, but it is a duplicated magic number with no UI control.
+This feature exposes it as a slider and removes the magic-number duplication.
+
+## Product decisions (confirmed with user, June 14, 2026)
+
+1. **Gate behavior: keep the downgrade chain.** Below threshold, skip the 3D
+   Three.js render and fall back to annotated photo, then unavailable. The
+   slider only moves the cutoff; it does not introduce a new hard-block state.
+2. **Slider apply: live re-render from cached analysis.** Moving the slider after
+   an analysis returns must re-evaluate the render mode against the
+   already-returned analysis JSON and re-render immediately. No new
+   `/analyze_image` or model call — zero added latency or Modal cost.
+3. **Source of truth: client authoritative.** The browser owns the threshold.
+   The client recomputes the render mode from the raw `confidence` plus the
+   slider value, overriding any `render_mode` in the server payload. Rendering is
+   already deterministic browser-side, so this is the minimal change. Per
+   `SECURITY.md`, the threshold is a non-credential UX control, so client
+   authority is acceptable; it is a quality gate, not a security gate.
+
+## Implementation plan (for Codex)
+
+All primary changes are in `index.html`. No backend signature change is required.
+
+### 1. Add the slider UI
+
+- Add a labeled `range` input to the viewport toolbar
+  (`index.html:550` `<div class="toolbar">`) or the readout panel near the
+  confidence metric (`index.html:577` `metric-row`). Recommended: toolbar, next
+  to `resetViewButton`, so it sits with the other live viewport controls.
+- Suggested markup:
+  ```html
+  <label class="threshold-control" for="confidenceThreshold">
+    Min confidence
+    <input id="confidenceThreshold" class="threshold-slider" type="range"
+           min="0" max="1" step="0.05" value="0.5"
+           aria-describedby="thresholdValue">
+    <span id="thresholdValue" aria-live="polite">50%</span>
+  </label>
+  ```
+- Default value `0.5` to preserve current behavior exactly.
+- Style consistent with existing `.tool-button` / toolbar aesthetic; keep it
+  keyboard-accessible (range inputs are by default) and screen-reader labeled,
+  matching the accessibility work already done in this repo (live status,
+  keyboard drop zone).
+
+### 2. Wire the threshold into render-mode selection
+
+- Introduce a single source for the current threshold, e.g.
+  `let confidenceThreshold = 0.5;`, updated from the slider's `input` event.
+- Update the slider value label (`#thresholdValue`) on `input`.
+- Replace the hardcoded `0.5` comparisons in `chooseRenderMode()`
+  (`index.html:760-772`) with `confidenceThreshold`:
+  - `confidence < confidenceThreshold` for the low-confidence branch.
+  - `confidence >= confidenceThreshold` for the `three` branch.
+- Make `renderScenePayload()` (`index.html:748`) **client authoritative**:
+  always compute the render mode via `chooseRenderMode(analysis)` using the
+  current threshold, instead of trusting `payload.render_mode`. Keep the
+  geometry/annotation capability checks (`hasUsableGeometry`, `hasAnnotations`)
+  so a high threshold never forces a 3D render that lacks geometry.
+
+### 3. Live re-render on slider change
+
+- Cache the last analysis (already stored at `index.html:722`
+  `window.lastAnalysis`) and the last scene payload.
+- On slider `change` (or debounced `input`), if an analysis is present and the
+  pipeline is not mid-request, call `renderScenePayload(lastScenePayload)` (or a
+  small `reevaluateRender()` helper that reads `window.lastAnalysis`) to rebuild
+  the scene from cached data. Do **not** call `/analyze_image` or
+  `/generate_scene` again — re-use the cached analysis JSON.
+- Guard against re-render while `setBusy(true)` is active to avoid racing an
+  in-flight request.
+- Reset/standby state should disable or ignore the slider re-render until an
+  analysis exists.
+
+### 4. Remove the magic-number duplication (recommended cleanup)
+
+Because decision #3 makes the client authoritative, the server's
+`select_render_mode()` `0.5` is now advisory only. Two acceptable options:
+
+- **Minimal:** leave `schema.py` `select_render_mode()` as-is (still returns a
+  reasonable default `render_mode`); the client overrides it. Add a code comment
+  noting the client is authoritative for the user-facing threshold.
+- **Cleaner (preferred if time allows):** extract the `0.5` default into a single
+  named constant, e.g. `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` in `schema.py`, and a
+  matching JS constant in `index.html`, so the default lives in one obvious place
+  per layer. Do **not** add a backend signature change to pass the slider value
+  to the server (decision #3 keeps the threshold client-side).
+
+## Out of scope / explicitly NOT doing
+
+- No new server endpoint or `/generate_scene` signature change (client
+  authoritative per decision #3).
+- No hard-block "confidence too low" state — the downgrade chain stays
+  (decision #1).
+- No re-running model inference when the slider moves (decision #2).
+- No model-authored HTML/JS/markup injection; rendering stays deterministic
+  Three.js from validated JSON (`SECURITY.md` Agent Guidance).
+
+## Implementation result
+
+- Added a toolbar confidence threshold slider in `index.html`, defaulting to
+  `0.5` / `50%`.
+- The browser now recomputes render mode from the cached analysis confidence and
+  current slider value, overriding advisory server `render_mode`.
+- Slider input re-renders from `window.lastScenePayload` only; it does not call
+  `/analyze_image` or `/generate_scene`.
+- Added `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` in `snap2sim/schema.py` for the
+  server's advisory fallback.
+- Local verification passed: schema/parser check, FastAPI `TestClient` root /
+  `/analyze_image` / `/generate_scene`, browser slider downgrade/promote with
+  no network calls, annotated-photo fallback, keyboard slider operation, mobile
+  no horizontal overflow, and canvas pointer targeting after normal overlay
+  hiding.
+
+## Verification checklist
+
+- Slider defaults to `0.5`; with default value, behavior matches current
+  production exactly (regression check).
+- Raising the threshold above a returned analysis's confidence downgrades a
+  previously-3D render to annotated photo, then to unavailable, live, with no
+  network call (confirm via browser devtools Network tab — no new
+  `/analyze_image` or `/generate_scene` requests on slider move).
+- Lowering the threshold promotes an annotated/unavailable result back to the 3D
+  cutaway, provided usable geometry exists.
+- Slider value label updates and is announced (`aria-live`).
+- Slider is keyboard-operable and does not steal focus from / block the canvas
+  OrbitControls (watch for the pointer-events class of bug fixed in the
+  REVIEW2.md pass).
+- Mobile layout: slider does not introduce horizontal overflow; toolbar still
+  fits.
+- `INFERENCE_BACKEND=local` sample mode still renders the example analysis with
+  the slider present.
+- Run existing local checks: schema/parser checks and the FastAPI `TestClient`
+  pass for `/`, `/analyze_image`, `/generate_scene`.
+
+## Touch points (file/line reference)
+
+- `index.html:550` toolbar — add slider markup.
+- `index.html:577` metric-row / `index.html:726` confidence display — optional
+  co-location with confidence readout.
+- `index.html:748` `renderScenePayload()` — make client authoritative.
+- `index.html:760` `chooseRenderMode()` — replace `0.5` with slider value.
+- `index.html:602-622` element refs + `index.html:624` state vars — add slider
+  element ref and `confidenceThreshold` state.
+- `snap2sim/schema.py:218` `select_render_mode()` — optional constant extraction.
+- `snap2sim/backend.py:47` `generate_scene()` — no change required.

README.md

@@ -128,14 +128,19 @@ Runtime flow:
 4. Browser renders deterministic Three.js primitives when geometry is usable,
    or overlays text-only callouts on the uploaded photo when the model only has
    image-space annotations.
-5. `/generate_scene` returns a validated
+5. A browser-side confidence slider controls the minimum analysis confidence
+   needed for the 3D cutaway. Moving it re-renders from cached analysis data
+   and does not call the model again.
+6. `/generate_scene` returns a validated
    `{ "renderer": "...", "render_mode": "...", "analysis": ... }` descriptor
    instead of model-authored HTML.
 
 The shell uses Chakra Petch and Fira Code from Bunny Fonts, an asymmetric
 63/37 viewport/readout split, a blueprint grid, amber/cyan instrument-panel
 colors, explicit Modal cold-start messaging, source-photo preview, and a
-play/pause control.
+play/pause control. The confidence slider defaults to 50%, matching the
+server's advisory fallback threshold, and the browser is authoritative for the
+visible render mode.
 
 The browser no longer injects model-authored HTML into the DOM. The model's
 job is limited to the structured analysis JSON contract in `snap2sim/schema.py`.

index.html

@@ -210,11 +210,13 @@
     .toolbar {
       position: absolute;
       left: 18px;
+      right: 18px;
       top: 18px;
       z-index: 10;
       display: flex;
       flex-wrap: wrap;
       gap: 8px;
+      max-width: calc(100% - 36px);
     }
 
     .tool-button {
@@ -241,6 +243,43 @@
       cursor: default;
     }
 
+    .threshold-control {
+      min-height: 34px;
+      display: flex;
+      align-items: center;
+      gap: 8px;
+      border: 1px solid var(--amber-dim);
+      padding: 0 9px;
+      color: var(--text-muted);
+      background: rgba(15, 19, 24, 0.86);
+      font-size: 0.68rem;
+      font-weight: 600;
+      text-transform: uppercase;
+      white-space: nowrap;
+    }
+
+    .threshold-control:focus-within {
+      border-color: var(--amber);
+    }
+
+    .threshold-slider {
+      width: 104px;
+      accent-color: var(--amber);
+      cursor: pointer;
+    }
+
+    .threshold-slider:disabled {
+      cursor: default;
+      opacity: 0.45;
+    }
+
+    .threshold-value {
+      min-width: 32px;
+      color: var(--amber);
+      font-family: "Fira Code", monospace;
+      text-align: right;
+    }
+
     .viewport-hint {
       position: absolute;
       left: 18px;
@@ -551,6 +590,11 @@
         <button id="uploadButton" class="tool-button" type="button">Load</button>
         <button id="playButton" class="tool-button" type="button" disabled>Play</button>
         <button id="resetViewButton" class="tool-button" type="button" disabled>Reset view</button>
+        <label class="threshold-control" for="confidenceThreshold">
+          <span>Min confidence</span>
+          <input id="confidenceThreshold" class="threshold-slider" type="range" min="0" max="1" step="0.05" value="0.5" aria-describedby="thresholdValue" disabled>
+          <span id="thresholdValue" class="threshold-value" aria-live="polite">50%</span>
+        </label>
       </div>
       <div id="viewport" aria-hidden="true"></div>
       <div id="viewportHint" class="viewport-hint" hidden>Drag to orbit / scroll to zoom</div>
@@ -603,6 +647,8 @@
     const uploadButton = document.getElementById("uploadButton");
     const playButton = document.getElementById("playButton");
     const resetViewButton = document.getElementById("resetViewButton");
+    const confidenceThresholdInput = document.getElementById("confidenceThreshold");
+    const thresholdValue = document.getElementById("thresholdValue");
     const dropZone = document.getElementById("dropZone");
     const viewport = document.getElementById("viewport");
     const viewportHint = document.getElementById("viewportHint");
@@ -621,11 +667,15 @@
     const partsEl = document.getElementById("parts");
     const rawJsonEl = document.getElementById("rawJson");
 
+    const DEFAULT_CONFIDENCE_THRESHOLD = 0.5;
     let activeMode = "idle";
     let paused = false;
+    let requestBusy = false;
     let coldStartTimer = 0;
+    let thresholdRenderTimer = 0;
     let fallbackRuntime = null;
     let currentPreviewUrl = "";
+    let confidenceThreshold = DEFAULT_CONFIDENCE_THRESHOLD;
     const MAX_CLIENT_IMAGE_BYTES = 8 * 1024 * 1024;
 
     uploadButton.addEventListener("click", () => fileInput.click());
@@ -669,6 +719,10 @@
     resetViewButton.addEventListener("click", () => {
       if (fallbackRuntime && fallbackRuntime.resetView) fallbackRuntime.resetView();
     });
+    confidenceThresholdInput.addEventListener("input", () => {
+      updateConfidenceThreshold();
+      scheduleThresholdRender();
+    });
 
     async function runPipeline(file) {
       const validationError = validateFile(file);
@@ -712,8 +766,11 @@
       playButton.textContent = "Play";
       playButton.disabled = true;
       resetViewButton.disabled = true;
+      confidenceThresholdInput.disabled = true;
       viewportHint.hidden = true;
       retryButton.hidden = true;
+      window.lastAnalysis = null;
+      window.lastScenePayload = null;
       viewport.replaceChildren();
       dropZone.classList.add("hidden");
     }
@@ -747,7 +804,11 @@
 
     function renderScenePayload(payload) {
       const analysis = payload && payload.analysis ? payload.analysis : payload;
-      const renderMode = payload && payload.render_mode ? payload.render_mode : chooseRenderMode(analysis);
+      if (analysis) {
+        window.lastAnalysis = analysis;
+        window.lastScenePayload = payload;
+      }
+      const renderMode = chooseRenderMode(analysis);
       if (renderMode === "three" && hasUsableGeometry(analysis)) {
         buildDeterministicScene(analysis);
       } else if (renderMode === "annotate" && hasAnnotations(analysis)) {
@@ -759,10 +820,10 @@
 
     function chooseRenderMode(analysis) {
       const confidence = typeof analysis.confidence === "number" ? analysis.confidence : 1;
-      if (hasAnnotations(analysis) && (confidence < 0.5 || !hasUsableGeometry(analysis))) {
+      if (hasAnnotations(analysis) && (confidence < confidenceThreshold || !hasUsableGeometry(analysis))) {
         return "annotate";
       }
-      if (hasUsableGeometry(analysis) && confidence >= 0.5) {
+      if (hasUsableGeometry(analysis) && confidence >= confidenceThreshold) {
         return "three";
       }
       if (hasAnnotations(analysis)) {
@@ -808,6 +869,7 @@
       playButton.textContent = "Pause";
       playButton.disabled = false;
       resetViewButton.disabled = false;
+      retryButton.hidden = true;
       viewportHint.hidden = false;
       viewport.replaceChildren();
       const stage = document.createElement("div");
@@ -939,6 +1001,7 @@
       playButton.textContent = "Play";
       playButton.disabled = true;
       resetViewButton.disabled = true;
+      retryButton.hidden = true;
       viewportHint.hidden = true;
       viewport.replaceChildren();
 
@@ -1194,8 +1257,10 @@
     }
 
     function setBusy(active) {
+      requestBusy = active;
       progress.classList.toggle("active", active);
       uploadButton.disabled = active;
+      confidenceThresholdInput.disabled = active || !window.lastScenePayload;
     }
 
     function setStatus(message, error) {
@@ -1203,6 +1268,23 @@
       statusEl.classList.toggle("error", Boolean(error));
     }
 
+    function updateConfidenceThreshold() {
+      const nextThreshold = Number(confidenceThresholdInput.value);
+      confidenceThreshold = Number.isFinite(nextThreshold)
+        ? nextThreshold
+        : DEFAULT_CONFIDENCE_THRESHOLD;
+      thresholdValue.textContent = Math.round(confidenceThreshold * 100) + "%";
+    }
+
+    function scheduleThresholdRender() {
+      window.clearTimeout(thresholdRenderTimer);
+      thresholdRenderTimer = window.setTimeout(() => {
+        if (!requestBusy && window.lastScenePayload) {
+          renderScenePayload(window.lastScenePayload);
+        }
+      }, 90);
+    }
+
     function revealScan() {
       scanLine.classList.remove("active");
       void scanLine.offsetWidth;

snap2sim/schema.py

@@ -213,10 +213,11 @@ ANALYSIS_SCHEMA: dict[str, Any] = {
 _SHAPES = {"box", "cylinder", "sphere", "gear", "rod"}
 _MOTIONS = {"rotate", "translate", "oscillate", "static"}
 _RENDER_MODES = {"three", "annotate", "unavailable"}
+DEFAULT_CONFIDENCE_THRESHOLD = 0.5
 
 
 def select_render_mode(analysis: dict[str, Any]) -> str:
-    """Pick the safest renderer for a validated analysis payload."""
+    """Pick the safest advisory renderer for a validated analysis payload."""
     explicit_mode = analysis.get("render_mode")
     if explicit_mode in _RENDER_MODES:
         return str(explicit_mode)
@@ -225,7 +226,7 @@ def select_render_mode(analysis: dict[str, Any]) -> str:
     has_geometry = any(isinstance(part, dict) and isinstance(part.get("geometry"), dict) for part in parts)
     has_annotation = any(isinstance(part, dict) and isinstance(part.get("annotation"), dict) for part in parts)
     confidence = analysis.get("confidence", 1)
-    low_confidence = _is_number(confidence) and float(confidence) < 0.5
+    low_confidence = _is_number(confidence) and float(confidence) < DEFAULT_CONFIDENCE_THRESHOLD
     if has_annotation and (low_confidence or not has_geometry):
         return "annotate"
     if has_geometry and not low_confidence: