Safety fix: never return an edible verdict from a single photo

The live build has a 0.85-confidence shortcut that can return an edible verdict from one photo. In testing, two photos of toxic mushrooms cleared it as SAFE. This PR removes the shortcut, so an edible verdict always needs a second photo, and brings the Space up to date with the current pipeline.

Merging is all that's needed. From HomesteaderLabs (alex@homesteaderlabs.com).

.github/workflows/observability.yml

@@ -0,0 +1,56 @@
+name: observability
+
+on:
+  push:
+    paths:
+      - "observability/**"
+      - "pipeline/**"
+      - ".github/workflows/observability.yml"
+  pull_request:
+    paths:
+      - "observability/**"
+      - "pipeline/**"
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:16
+        env:
+          POSTGRES_PASSWORD: forager
+          POSTGRES_DB: forager_fs_obs
+        ports:
+          - 5434:5432
+        options: >-
+          --health-cmd "pg_isready -U postgres"
+          --health-interval 2s
+          --health-timeout 5s
+          --health-retries 15
+    env:
+      FORAGER_OBS_DSN: postgresql://postgres:forager@localhost:5434/forager_fs_obs
+    steps:
+      # Two checkouts side by side, because observability/requirements.txt
+      # installs the shared package as `-e ../forager-obs`. Actions cannot check
+      # out above the workspace, so this repo goes in a subdirectory too and the
+      # later steps set working-directory.
+      - uses: actions/checkout@v4
+        with:
+          path: forager-field-station
+      - uses: actions/checkout@v4
+        with:
+          repository: thefullnacho/forager-obs
+          path: forager-obs
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+      - name: Install deps (incl. editable forager-obs)
+        working-directory: forager-field-station
+        run: pip install -r observability/requirements.txt
+      - name: Shared safety rules (no DB, no models)
+        working-directory: forager-obs
+        run: python -m pytest -q
+      - name: Two-photo safety tests + observability schema/views
+        working-directory: forager-field-station
+        run: |
+          python -m pytest observability/tests/ tests/test_two_photo.py -q

.pytest_cache/.gitignore

@@ -0,0 +1,2 @@
+# Created by pytest automatically.
+*

.pytest_cache/CACHEDIR.TAG

@@ -0,0 +1,4 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by pytest.
+# For information about cache directory tags, see:
+#	https://bford.info/cachedir/spec.html

.pytest_cache/README.md

@@ -0,0 +1,8 @@
+# pytest cache directory #
+
+This directory contains data from the pytest's cache plugin,
+which provides the `--lf` and `--ff` options, as well as the `cache` fixture.
+
+**Do not** commit this to version control.
+
+See [the docs](https://docs.pytest.org/en/stable/how-to/cache.html) for more information.

.pytest_cache/v/cache/nodeids

@@ -0,0 +1,13 @@
+[
+  "observability/tests/test_observability.py::test_abstain_is_never_toxic_as_edible",
+  "observability/tests/test_observability.py::test_positive_recall_view",
+  "observability/tests/test_observability.py::test_safety_view_reports_zero_for_safe_handling",
+  "observability/tests/test_observability.py::test_writer_derives_toxic_as_edible",
+  "tests/test_two_photo.py::test_confident_deadly_mushroom_is_not_shown_safe",
+  "tests/test_two_photo.py::test_deadly_can_surface_on_single_photo",
+  "tests/test_two_photo.py::test_single_photo_edible_only_when_confident",
+  "tests/test_two_photo.py::test_single_photo_never_commits_edible",
+  "tests/test_two_photo.py::test_single_photo_never_surfaces_edible",
+  "tests/test_two_photo.py::test_two_agreeing_photos_can_commit_safe",
+  "tests/test_two_photo.py::test_two_photos_no_deadly_shown_edible"
+]

STATUS.md

@@ -0,0 +1,79 @@
+# STATUS
+
+Running log of where the Field Station actually is. Newest entry at the top. Append, do not
+rewrite.
+
+Cross-repo facts live in the Forager wiki at `~/Documents/Forager/forager-wiki/`.
+
+---
+
+## 2026-08-15 — shared observability package; the dev harness stops shipping to the Space
+
+**The dev harness was being uploaded to the public Space.** `deploy.py`'s IGNORE list excluded
+`scripts/*` but not `observability/*`, so the whole Postgres harness went up with every deploy.
+Now excluded. Verified the Space runtime never imported it: `app.py`, `pipeline/` and `game/`
+are clean.
+
+**`forager-obs` adopted.** The `toxic_as_edible` rule was implemented here inline in
+`SessionWriter` and separately in forager_ml as a property on `InferenceOutcome`, with the
+flagship 0.0 claim resting on both. It now comes from
+`github.com/thefullnacho/forager-obs` (public, `main`), an editable sibling install, along with
+connect/migrate, the batch-writer contract (`SessionWriter` is now a `BaseWriter`) and the
+val-set walk including `k_groups`.
+
+**The session-grained schema stays here.** forager_ml's is image-grained. Those answer
+different questions and are deliberately not merged — recorded in the wiki so a later pass
+does not "fix" it.
+
+**Compose collision fixed.** This repo and forager_ml both bound host port 5433 *and* derived
+the same Compose project name from their parent directory (`observability` in both), so
+bringing up the second silently recreated the first's container. Now 5434 /
+`forager-fs-obs`, and both run side by side for the first time.
+
+**Verified:** 4 observability tests + 4 two-photo tests green against a live Postgres on 5434,
+and the Space's pipeline imports unchanged.
+
+### In flight
+
+- **`two-photo-safety` branch is 1 commit ahead and this repo has no git remote.** It is a
+  local repo that deploys to Hugging Face via `deploy.py`.
+- **The Space has not been redeployed.** The `deploy.py` exclusion takes effect on the next
+  `python deploy.py <user>/forager-field-station`, so `observability/` is still present in the
+  currently published Space.
+
+### Next concrete action
+
+Decide whether to redeploy the Space now (which removes the dev harness from the public repo)
+or fold it into the next content deploy.
+
+## 2026-10-25 — merge + GitHub remote
+
+**Changed:** `two-photo-safety` fast-forwarded into `master` (two-photo safety protocol,
+observability harness, confidence-banded single-photo gate, shared `forager-obs`). Repo now has
+a private GitHub remote, `thefullnacho/forager-field-station`, default branch `master`; both
+branches pushed. `assets/` (local backyard photos, video) is gitignored.
+
+### In flight
+
+- **The Space has still not been redeployed.** `observability/` is still in the published
+  Space until the next `deploy.py` run. [non-production] Approve the redeploy (it publishes).
+- The `forager_ml` convergence drift check stays local-only. Its reason (no remote here) is
+  gone, so it could move into CI now.
+
+### Next concrete action
+
+Space redeploy deferred (2026-10-25): fold it into the next content deploy, which will also
+remove `observability/` from the public Space.
+
+## 2026-10-27 — safety fix deploy blocked
+
+The fix that removes the 0.85 single-photo edible shortcut (`209f6b6`) is on master but **not
+live**. The live Space is `build-small-hackathon/Forager-Field-Notes`, not the
+`HomesteaderLabs/forager-field-station` named in the docs (that one doesn't exist). The
+HomesteaderLabs token has read-only access to the hackathon org, so the upload was refused (403).
+Nothing changed on the Space; it still serves the 2026-06-14 build and is sleeping. A new Space
+under HomesteaderLabs returned 402: Gradio on free CPU needs PRO.
+
+- [non-production] Get write access back on `build-small-hackathon`, or choose a new home
+  (PRO for HomesteaderLabs, or a different host), then re-run `deploy.py`.
+- [non-production] Until then, decide whether the live Space should keep serving the shortcut.

app.py

@@ -15,7 +15,8 @@ import time
 
 import gradio as gr
 
-from pipeline.convergence import build_result
+from pipeline.convergence import build_result  # noqa: F401  (re-exported for callers)
+from pipeline.session import resolve_session
 from pipeline.infer import Pipeline
 from game.ui import build_game_tab, build_stump_tab
 
@@ -73,7 +74,17 @@ def _loading() -> str:
 def _card(r) -> str:
     label, color, glyph = TIER.get(r.safety, TIER["UNKNOWN"])
 
-    if r.abstained:
+    if r.abstained and r.reason == "need_second_photo":
+        # Leaning toward an edible ID, but the safety rule withholds it until a
+        # second photo confirms. Not a failure — a request.
+        color, label, glyph = INK_UNK, "SECOND PHOTO", "◎"
+        title, sub = "ADD A SECOND ANGLE", "an edible ID needs two photos that agree"
+        rows = [
+            f"<div class='rdt-row'>{r.key_diff}</div>",
+            f"<div class='rdt-meta'>photograph the same specimen from another angle, "
+            f"then scan again</div>",
+        ]
+    elif r.abstained:
         color, label, glyph = INK_UNK, "UNKNOWN", "?"
         title, sub = "UNKNOWN", "not confident enough — refusing by default"
         rows = [
@@ -112,14 +123,20 @@ def _card(r) -> str:
     )
 
 
-def identify(image):
-    """Generator: show the vine animation, then the readout."""
-    if image is None:
+def identify(image, image2=None):
+    """Generator: show the vine animation, then the readout.
+
+    Takes an optional second photo. The safety rule (pipeline.session) means a
+    single photo never surfaces an edible verdict — it asks for the second — so
+    both inputs feed one resolve_session call.
+    """
+    images = [im for im in (image, image2) if im is not None]
+    if not images:
         yield _idle()
         return
     t0 = time.time()
     yield _loading()
-    result = build_result(PIPE.identify(image))
+    result = resolve_session(PIPE, images)
     elapsed = time.time() - t0
     if elapsed < 1.6:                       # let the vine finish drawing
         time.sleep(1.6 - elapsed)
@@ -358,6 +375,9 @@ with gr.Blocks(title="Forager's Field Station") as demo:
                 with gr.Column(scale=1):
                     img = gr.Image(type="pil", label="SPECIMEN", sources=["upload", "webcam"],
                                    elem_classes="eink-input", height=300)
+                    img2 = gr.Image(type="pil", label="SECOND ANGLE — confirms an edible ID",
+                                    sources=["upload", "webcam"], elem_classes="eink-input",
+                                    height=180)
                     btn = gr.Button("▸ SCAN SPECIMEN", variant="primary", elem_classes="eink-scan")
                     if os.path.isdir(EXAMPLES_DIR):
                         samples = [[os.path.join(EXAMPLES_DIR, f)] for f in (
@@ -371,8 +391,9 @@ with gr.Blocks(title="Forager's Field Station") as demo:
             gr.HTML(QUOTE_BAR)
             gr.Markdown(SAFETY_NOTICE, elem_id="notice")
 
-            btn.click(identify, inputs=img, outputs=out)
-            img.change(identify, inputs=img, outputs=out)
+            btn.click(identify, inputs=[img, img2], outputs=out)
+            img.change(identify, inputs=[img, img2], outputs=out)
+            img2.change(identify, inputs=[img, img2], outputs=out)
 
         with gr.Tab("⚔ BEAT THE MACHINE"):
             build_game_tab(PIPE)

pipeline/convergence.py

@@ -37,6 +37,9 @@ class ForagerResult:
     expert_model:    str
     abstained:       bool
     reason:          str     # why we abstained (or "" when committed)
+    energy:          float | None = None  # winning call's OOD energy — telemetry,
+                                          # not a gate (energy doesn't separate
+                                          # deadly-as-safe); kept for drift monitoring.
 
     @property
     def is_deadly(self) -> bool:
@@ -71,7 +74,7 @@ def _commit(domain: str, c: dict) -> ForagerResult:
         domain=domain, species=species, scientific_name=meta["scientific"], confidence=conf,
         safety=meta["safety"], lookalike=meta["lookalike"], key_diff=meta["key_diff"],
         low_confidence=conf < LOW_CONFIDENCE_THRESHOLD, expert_model=c["expert"],
-        abstained=False, reason="",
+        abstained=False, reason="", energy=c.get("energy"),
     )
 
 

pipeline/infer.py

@@ -122,6 +122,11 @@ class Pipeline:
         idx = int(probs.argmax())
         return self._classes[name][idx], float(probs[idx]), _energy(logits)
 
+    def _run_probs(self, name: str, x: np.ndarray) -> tuple[np.ndarray, float]:
+        """Returns (full softmax prob vector, energy) — for multi-photo fusion."""
+        logits = self._sessions[name].run(None, {"input": x})[0][0]
+        return _softmax(logits), _energy(logits)
+
     def identify(self, img: Image.Image) -> dict:
         """
         Returns a dict describing the call:
@@ -148,3 +153,55 @@ class Pipeline:
 
         call = {"expert": ename, "species": species, "confidence": conf, "energy": round(energy, 4)}
         return {**out, "abstain": False, "calls": [call]}
+
+    def identify_many(self, images: list[Image.Image]) -> dict:
+        """Fuse K photos of one specimen into a single call.
+
+        Late-fusion by GEOMETRIC MEAN of the experts' softmax vectors (treats
+        each photo as independent evidence: agreement sharpens the peak,
+        disagreement flattens it), plus a CONSENSUS requirement — every photo's
+        own top class must match the fused top class. This is the K>=2 protocol
+        validated in scripts/multiview_smoketest.py (deadly-shown-edible -> 0.00%
+        at K=2). K=1 degenerates exactly to identify()'s single call.
+
+        Abstains (and says why) when the photos disagree on domain or class, so
+        a confident-but-wrong single frame can't carry an edible verdict.
+        """
+        if not images:
+            return {"domain": "unknown", "domain_confidence": 0.0,
+                    "abstain": True, "reason": "low_confidence"}
+        if len(images) == 1:
+            return self.identify(images[0])
+
+        # ── Stage 1: every photo must route to the SAME in-domain domain ──────
+        routed = [self._run(ROUTER, preprocess(img)) for img in images]
+        domains = {d for d, _, _ in routed}
+        dconf = min(c for _, c, _ in routed)
+        domain = next(iter(domains))
+        out = {"domain": domain if len(domains) == 1 else "mixed",
+               "domain_confidence": dconf}
+        if len(domains) != 1 or dconf < ROUTER_CONFIDENCE_THRESHOLD or domain not in DOMAIN_EXPERTS:
+            reason = ("off_domain" if len(domains) == 1 and domain not in DOMAIN_EXPERTS
+                      else "uncertain_domain")
+            return {**out, "abstain": True, "reason": reason}
+
+        # ── Stage 2: run the domain's expert on each photo, fuse, check consensus
+        ename = DOMAIN_EXPERTS[domain]
+        classes = self._classes[ename]
+        per_photo = [self._run_probs(ename, preprocess(img)) for img in images]
+        prob_rows = np.stack([p for p, _ in per_photo])           # [K, C]
+        energy = float(np.mean([e for _, e in per_photo]))
+
+        logp = np.log(np.clip(prob_rows, 1e-9, 1.0)).mean(axis=0)  # geometric mean
+        fused = np.exp(logp - logp.max())
+        fused /= fused.sum()
+        top = int(fused.argmax())
+
+        per_photo_top = {int(row.argmax()) for row in prob_rows}
+        if per_photo_top != {top}:
+            # Photos disagree on the species — refuse rather than commit.
+            return {**out, "abstain": True, "reason": "no_consensus"}
+
+        call = {"expert": ename, "species": classes[top],
+                "confidence": float(fused[top]), "energy": round(energy, 4)}
+        return {**out, "abstain": False, "calls": [call]}

pipeline/session.py

@@ -0,0 +1,64 @@
+"""session.py — the two-photo safety policy that sits over the resolver.
+
+The core safety rule, stated plainly:
+
+    You are never shown an edible verdict on the strength of a single photo.
+
+A single photo can show DEADLY (over-warning is always safe) or abstain. A
+SAFE/CAUTION identification is only surfaced when a SECOND photo agrees with the
+first (geometric-mean fusion + consensus, in infer.identify_many). So the second
+photo is requested exactly, and only, when the system is leaning toward telling
+you something is edible — the one moment extra rigor is worth the friction.
+
+This is what closes the ~1% single-shot deadly-shown-edible residual to 0.00%
+(scripts/multiview_smoketest.py).
+"""
+
+from PIL import Image
+
+from .convergence import ForagerResult, build_result
+from .infer import Pipeline
+
+# There is no confidence bar that lets one photo commit edible. A 0.85 bar used to:
+# the val sets had put every single-photo deadly-as-edible miss at <= 0.772. Over
+# the full val sets, two toxic mushroom photos commit as SAFE above it (lion's
+# mane at 0.91, chanterelle at 0.87), and val is the optimistic case. A confident
+# wrong answer looks exactly like a confident right one, so no threshold on one
+# photo's confidence can hold; only a second look can.
+
+
+def resolve_session(pipe: Pipeline, images: list[Image.Image]) -> ForagerResult:
+    """Run the K-photo pipeline and apply the single-photo edible gate.
+
+    - 0 photos -> abstain.
+    - DEADLY (any photo count) -> surface it. Over-warning is safe.
+    - 1 photo, SAFE/CAUTION, at any confidence -> withhold; ask for a second photo.
+    - 2+ photos -> the fused/consensus verdict from identify_many stands.
+    """
+    result = build_result(pipe.identify_many(images))
+
+    if (len(images) == 1
+            and not result.abstained
+            and result.safety in ("SAFE", "CAUTION")):
+        return _need_second_photo(result)
+    return result
+
+
+def _need_second_photo(provisional: ForagerResult) -> ForagerResult:
+    """Turn a single-photo edible lean into a 'add a second photo' refusal,
+    carrying the provisional species so the UI can say what it's leaning toward
+    without committing to it."""
+    return ForagerResult(
+        domain=provisional.domain,
+        species="unknown",
+        scientific_name=provisional.scientific_name,
+        confidence=provisional.confidence,
+        safety="UNKNOWN",
+        lookalike=provisional.lookalike,
+        key_diff=(f"Leaning {provisional.species.replace('_', ' ')}, but an edible call "
+                  f"needs a second photo from a different angle to confirm."),
+        low_confidence=True,
+        expert_model=provisional.expert_model,
+        abstained=True,
+        reason="need_second_photo",
+    )

tests/test_two_photo.py

@@ -0,0 +1,108 @@
+"""Two-photo safety protocol tests, against the real ONNX models.
+
+The load-bearing guarantee: no edible verdict is ever surfaced from a single
+photo, and two disagreeing photos abstain rather than commit.
+"""
+
+import os
+import sys
+
+import pytest
+from PIL import Image
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
+
+from pipeline.infer import Pipeline  # noqa: E402
+from pipeline.session import resolve_session  # noqa: E402
+
+ML = "/home/alex/Documents/Forager/forager_ml"
+
+
+@pytest.fixture(scope="module")
+def pipe():
+    return Pipeline()
+
+
+def _img(path):
+    return Image.open(path)
+
+
+def _first(cls_dir, n=1):
+    files = sorted(f for f in os.listdir(cls_dir)
+                   if f.lower().endswith((".jpg", ".jpeg", ".png")))
+    return [os.path.join(cls_dir, f) for f in files[:n]]
+
+
+# ── Policy: single photo never surfaces an edible verdict ────────────────────
+
+def test_single_photo_never_commits_edible(pipe):
+    """A single photo never surfaces an edible (SAFE/CAUTION) verdict, at any
+    confidence: it is withheld and routed to a second photo."""
+    safe_dir = f"{ML}/berry_dataset_split/val/blackberry_common"
+    if not os.path.isdir(safe_dir):
+        pytest.skip("val data not present")
+    for path in _first(safe_dir, 12):
+        r = resolve_session(pipe, [_img(path)])
+        assert not (not r.abstained and r.safety in ("SAFE", "CAUTION")), (
+            f"single photo committed {r.safety} at {r.confidence:.3f} for {path}")
+        if r.abstained:
+            assert r.reason in ("need_second_photo", "low_confidence",
+                                "uncertain_domain", "off_domain")
+
+
+def test_confident_deadly_mushroom_is_not_shown_safe(pipe):
+    """The case that removed the 0.85 shortcut: toxic-catch-all photos the
+    mushroom expert calls lion's mane (0.91) and chanterelle (0.87)."""
+    d = f"{ML}/high_value_dataset_split/val/high_value_toxics"
+    paths = [f"{d}/599814795.jpg", f"{d}/616602478.jpg"]
+    if not all(os.path.isfile(p) for p in paths):
+        pytest.skip("val data not present")
+    for path in paths:
+        r = resolve_session(pipe, [_img(path)])
+        assert r.abstained and r.reason == "need_second_photo", (
+            f"{path} surfaced as {r.species} {r.safety} at {r.confidence:.3f}")
+
+
+def test_two_agreeing_photos_can_commit_safe(pipe):
+    """Two photos of the same SAFE class should be allowed to commit (the whole
+    point — the second photo unlocks an edible verdict)."""
+    safe_dir = f"{ML}/berry_dataset_split/val/blackberry_common"
+    if not os.path.isdir(safe_dir):
+        pytest.skip("val data not present")
+    committed = 0
+    paths = _first(safe_dir, 12)
+    # try a few pairs; at least one agreeing pair should commit SAFE
+    for i in range(0, len(paths) - 1, 2):
+        r = resolve_session(pipe, [_img(paths[i]), _img(paths[i + 1])])
+        if not r.abstained and r.safety == "SAFE":
+            committed += 1
+    assert committed >= 1, "no agreeing SAFE pair ever committed — fusion too strict"
+
+
+def test_deadly_can_surface_on_single_photo(pipe):
+    """Over-warning is safe: a DEADLY read on one photo is allowed through."""
+    deadly_dir = f"{ML}/berry_dataset_split/val/pokeweed_toxic"
+    if not os.path.isdir(deadly_dir):
+        pytest.skip("val data not present")
+    # We don't require every pokeweed to read DEADLY on one shot, only that the
+    # policy PERMITS a single-photo DEADLY (never converts it to need_second).
+    for path in _first(deadly_dir, 8):
+        r = resolve_session(pipe, [_img(path)])
+        if r.safety == "DEADLY":
+            assert not r.abstained
+            return
+    pytest.skip("no single-shot DEADLY in sample (still safe: they abstained)")
+
+
+def test_two_photos_no_deadly_shown_edible(pipe):
+    """The core claim: fuse two photos of DEADLY berry specimens; none may be
+    surfaced as edible (SAFE/CAUTION committed)."""
+    for cls in ("pokeweed_toxic", "canada_moonseed_deadly"):
+        d = f"{ML}/berry_dataset_split/val/{cls}"
+        if not os.path.isdir(d):
+            pytest.skip("val data not present")
+        paths = _first(d, 12)
+        for i in range(0, len(paths) - 1, 2):
+            r = resolve_session(pipe, [_img(paths[i]), _img(paths[i + 1])])
+            assert not (not r.abstained and r.safety in ("SAFE", "CAUTION")), (
+                f"{cls} pair surfaced as {r.safety} — toxic-as-edible at K=2")