verify: routing leg passes; dispatch blocked by private-Space 404 (HF_TOKEN unset)

Signed-in test against the orchestrator dev Space: it correctly classified a DNA
question and routed to PDAC Genomics Agent. Dispatch then failed with a repo-lookup
404 because this Space is private and the orchestrator sends no token — consistent
with the caller-side HF_TOKEN root cause. Nothing to fix in this repo.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

TODO.md

@@ -202,6 +202,39 @@ the design doc's M0–M4 is the intent behind it — see that doc's §0).
       statistical-honesty stance (no cross-gene correction, `testable=false` + reason, small-n
       caveats), and metadata-label vs live-PurIST provenance. No behaviour change.
 
+## Test-suite hermeticity ✅ (2026-07-28)
+
+- [x] **Enforce "no live network" instead of asserting it** — session-wide socket guard in
+      `tests/conftest.py` (module scope, so it covers *collection*; an autouse fixture would run
+      too late for import-time leaks). Non-loopback `connect`/`connect_ex`/`create_connection`
+      raise, naming the host. Loopback + AF_UNIX allowed.
+- [x] **Close the last hole: the three UI modules.** `test_ui_handler_gating` /
+      `test_deploy_surface` / `test_machine_api_auth` each `import gradio_ui`, whose Blocks graph
+      is built at module scope; off a Space, gradio's `attach_oauth` mocks the login button by
+      calling HF `whoami`. conftest stubs `gradio.oauth._get_mocked_oauth_info` — the narrowest
+      seam, and **test-only: `gradio_ui.py` is unchanged**, so the module-level `demo` that HF
+      Spaces launches still works. Restructuring `gradio_ui` was considered and rejected as
+      disproportionate.
+- [x] **`tests/test_hermetic.py`** — the guard is itself tested (installed / refuses / message
+      names the host / loopback still works / all three UI modules import). Without this the guard
+      decays into the same unchecked docstring it replaced.
+- [x] **Verified: 143 pass with sockets blocked outright, no `--ignore`** (was 111 with the three
+      UI modules ignored). Mutation-tested — reverting the `study_metadata` patch reproduces the
+      original bug and the guard catches it. Runtime 22s → 4s.
+- [x] **CI no longer needs an HF login** (previously those three modules could not import without
+      an HF token at all). Verified with `HF_TOKEN` unset.
+- [ ] **Watch on gradio upgrades:** the stub targets gradio's private
+      `_get_mocked_oauth_info`. If a bump renames it, `test_hermetic.py`'s UI-import test fails —
+      that failure is the signal, not a flake. Re-point the stub.
+- [ ] Optional: promote the stub-vs-restructure choice to an ADR if it proves contested.
+      **Next free is 0008** — `main` has 0006 (variant×subtype join) *and* 0007 (re-curation
+      cadence) published. Claim in `RESERVED.md` on `main`, alone, first.
+- [ ] ⚠️ **This branch's `docs/adr/RESERVED.md` is stale — it stops at 0005 and says "next free
+      0006", both of which are wrong.** It is the per-lane-copy failure the ledger's own postmortem
+      describes: the shared-row conflict cannot fire between branches, so `ls docs/adr/` and the
+      local ledger will both hand you a burned number. Check with
+      `git show main:docs/adr/RESERVED.md` before claiming, and rebase this branch.
+
 ## Open questions
 
 - ~~HF account / Space name~~ **RESOLVED 2026-07-24** — Space `anne-voigt/pdac-genomics-agent` created

memory.md

@@ -839,6 +839,7 @@ by running the suite with `socket.connect` blocked outright — **111 tests pass
 all** (the three UI test modules still touch it, but that is Gradio's mocked-OAuth HF call at
 import in a local dev env, not our code and not present on a Space).
 
+<<<<<<< HEAD
 
 ---
 
@@ -865,3 +866,111 @@ misled again. One contract, with instructions when you miss it.
 Ordering is gate-first: an unauthenticated caller gets `denied` and learns nothing about our
 inputs — pinned by `test_argument_errors_are_reported_only_after_the_auth_gate`. 132 tests (7 new,
 alongside the denial-diagnosis guards they mirror).
+=======
+---
+
+## 2026-07-28 — Hermetic is now *enforced*, and the whole suite is hermetic
+
+Finishes the item above. That fix patched the one hole that had been found; this one removes the
+reason holes could survive — and closes the last remaining one.
+
+**The lesson being encoded:** the "NO live network" claim lived in a docstring for weeks while
+being false. Nothing checked it. So the property is now enforced by a guard and the guard is
+itself under test. Do not "simplify" either away.
+
+- **Socket guard** (`tests/conftest.py`, module scope) — wraps `socket.socket.connect` /
+  `connect_ex` / `socket.create_connection` for the whole session and raises on any non-loopback
+  address, naming the host and pointing at conftest for the fix. Installed at *import* of
+  conftest, not as an autouse fixture, deliberately: the leaks that mattered happened at
+  **collection** time (a test module's `import gradio_ui` runs the app graph at module scope), and
+  a fixture runs too late to catch those. Loopback and AF_UNIX are allowed — local sockets are not
+  what the guard is about.
+- **The three UI modules no longer touch the network.** `test_ui_handler_gating.py`,
+  `test_deploy_surface.py`, `test_machine_api_auth.py` all `import gradio_ui`, whose `gr.Blocks`
+  graph is built at module scope. Off a Space, gradio's `attach_oauth` falls back to
+  `_add_mocked_oauth_routes`, which calls HF `whoami` to mock a local login profile. **That is
+  gradio's debug convenience, not our code**, and it does not happen on the deployed Space (there
+  `SPACE_ID` is set and the real OAuth routes are used). conftest now stubs
+  `gradio.oauth._get_mocked_oauth_info`.
+  - **Why that seam and not the others considered:** it is the only function on that path that
+    touches the network, so stubbing it leaves every route gradio registers intact and the UI
+    tests still exercise the real graph. Restructuring `gradio_ui.py` to build Blocks inside a
+    function was rejected as far more invasive for a test-only problem — and its module-level
+    `demo` is what HF Spaces launches, so it must keep working. **`gradio_ui.py` is unchanged;
+    the entire fix is test-only.**
+  - **Known coupling:** `_get_mocked_oauth_info` is a gradio private. A gradio upgrade that
+    renames it would break the stub — which is why `test_hermetic.py` imports all three UI
+    modules explicitly, so that breakage surfaces as a named test failure rather than as a
+    mystery connection at collection time.
+- **`tests/test_hermetic.py`** (new, 7 tests) — asserts the guard is installed, that it refuses a
+  routable address, that the message names the host *and* points at the fix (a guard with an
+  unhelpful message gets disabled by the next person who trips it), that loopback still works, and
+  that the three UI modules import cleanly.
+- **Bonus: CI no longer needs an HF login.** `_get_mocked_oauth_info` raises `ValueError` when the
+  machine has no HF token, so those three modules could not have been imported on a clean CI box
+  at all. Verified with `HF_TOKEN` unset and `HF_HOME` pointed at an empty dir.
+
+**Verified — 143 passed with sockets blocked outright and no `--ignore`** (136 before + 7 new),
+using the same method that exposed the original bug: stub `socket.socket.connect` /
+`socket.create_connection` to raise, then `pytest.main(["-q"])`. Previously 111 passed that way
+with the three UI modules ignored.
+
+**Mutation-tested, so the guard is not vacuous:** removing the `study_metadata` patch and the
+`_req` stub reproduces the original bug, and the guard catches it with
+`AssertionError: the test suite attempted a network connection to ('www.cbioportal.org', 443)`.
+Restored after.
+
+**Suite runtime 22s → 4s** — that difference *was* the network, on a good day. On a bad one it was
+the 54-second, 12-error run that started this.
+
+**No ADR.** The contested part (stub a gradio private vs. restructure `gradio_ui`) is test-only,
+reversible, and documented at both the seam and the test. If it should be one, **next free is
+0008** — `main` has 0006 (variant×subtype join) *and* 0007 (re-curation cadence) published — and
+per `RESERVED.md` the claim must land on `main` alone, first.
+
+⚠️ **This branch's `RESERVED.md` is stale: it stops at 0005 and states "next free 0006".** Both
+0006 and 0007 are burned. This is the third instance of the same failure the ledger's own
+postmortem describes — per-lane copies mean the one-shared-row conflict cannot fire between
+branches, so *the ledger in your working tree is exactly the thing you cannot trust for this*.
+Verify with `git show main:docs/adr/RESERVED.md` before claiming a number. (The rollup suggests
+`origin/main`, but `origin` here IS the HF Space; local `main` is the reliable read.) Found by
+reading `SHOWCASE_STATUS.md` before updating it — the same habit that caught collision #3.
+
+**Environment note:** this repo now has a venv at `../../.venv` (Python 3.14) created for this
+work — `gradio[oauth]==6.18.0` (the `oauth` extra is required or the UI modules fail to import at
+all), plus the rest of `requirements.txt` and pytest. Previously the memory said to borrow
+`DecoupleRpy_Agent/.venv`.
+>>>>>>> dbaeaa3 (tests: enforce hermeticity with a socket guard; last 3 network-touching modules closed)
+
+---
+
+## 2026-07-28 — Routing leg VERIFIED end-to-end; dispatch blocked by a private-Space 404
+
+The one thing the orchestrator registration task could not check headlessly — does the router
+actually *classify* a DNA question and dispatch here — is now verified in a signed-in browser
+session against the orchestrator **dev** Space (`5cbf1d9`).
+
+Asked: *"Which samples in the TCGA pancreatic cohort carry a KRAS G12D mutation, and what is the
+KRAS alteration frequency?"*
+
+**Classification: PASS.** The orchestrator answered *"Routing to PDAC Genomics Agent for
+computation."* — it recognised a DNA question and picked this specialist over the subtype and
+DE agents. That is the registration working, including the deliberate choice to drop bare `KRAS`
+from the trigger keywords (the hotspot allele `G12D` carried the intent instead).
+
+**Dispatch: FAIL, and the error names its own cause.** The call came back:
+
+> The specialist tool server could not be reached… `404 Client Error. Repository Not Found for
+> url: https://huggingface.co/api/spaces/anne-voigt/pdac-genomics-agent`. If you are trying to
+> access a private or gated repo, make sure you are authenticated…
+
+This **Space is PRIVATE**, so without credentials the orchestrator cannot even *resolve* the repo,
+let alone reach the gate. The failure is a repo-lookup 404, not an allow-list denial — consistent
+with the caller-side root cause already found: the orchestrator attaches its token only when
+`HF_TOKEN` is set, and that secret is unset on its Space.
+
+**Nothing to change here.** The fix is one secret on the *orchestrator's* Space, and it is a
+credential operation for a human. Worth noting the order of failure for whoever does it: with
+`HF_TOKEN` unset the symptom is a 404 (invisible private repo); once set, the next thing exercised
+is the allow-list — and the identity it resolves to must be on `ALLOWED_IDS` here
+(`anne-voigt,cpelz741` today).

tests/conftest.py

@@ -2,8 +2,16 @@
 
 Mirrors the sibling specialists: capture a few genes×samples of real API JSON once, commit
 it, and route the client's fetch functions through it so CI is hermetic.
+
+"Hermetic" is **enforced**, not asserted: this module blocks outbound sockets for the whole
+session (see below). It is enforced because the docstring alone was wrong for weeks — the
+fixture curated, curation fetched `study_metadata`, and nothing patched it, so every
+fixture-using test hit cbioportal.org. It surfaced as a run that failed with 12 errors and
+passed on retry: a third party's latency reported as our bug. A property no test checks is a
+property you do not have.
 """
 import json
+import socket
 from pathlib import Path
 
 import pytest
@@ -11,6 +19,107 @@ import pytest
 _FIXTURE = Path(__file__).parent / "fixtures" / "cbioportal_paad_mini.json"
 
 
+# --------------------------------------------------------------------------- #
+# Hermetic enforcement — no outbound sockets for the duration of the session
+# --------------------------------------------------------------------------- #
+# Installed at import of this module rather than as an autouse fixture, because the leaks we
+# need to catch happen at *collection* time (a test module's `import gradio_ui` runs the whole
+# app graph at module scope). conftest is imported before collection; a fixture would be too
+# late.
+
+_LOOPBACK = frozenset({"127.0.0.1", "::1", "localhost", "", "0.0.0.0"})
+
+_real_connect = socket.socket.connect
+_real_connect_ex = socket.socket.connect_ex
+_real_create_connection = socket.create_connection
+
+
+def _is_local(address) -> bool:
+    """Loopback and AF_UNIX are allowed; everything routable is not.
+
+    Local sockets are how a test server or an ASGI transport talks to itself — legitimate, and
+    not what this guard is about. The guard is about reaching a *third party*.
+    """
+    if isinstance(address, (str, bytes)):  # AF_UNIX path
+        return True
+    if isinstance(address, tuple) and address:
+        host = address[0]
+        if isinstance(host, bytes):
+            host = host.decode("utf-8", "replace")
+        return str(host) in _LOOPBACK
+    return False
+
+
+def _blocked(address):
+    raise AssertionError(
+        f"the test suite attempted a network connection to {address!r}.\n"
+        "This suite is hermetic by contract: fixtures are pinned JSON captures and the request "
+        "path reads committed curated artifacts (ADR-0005 C4). A real connection here means "
+        "something is unpatched — patch it in tests/conftest.py rather than relaxing this "
+        "guard. Loopback is allowed if you genuinely need a local server."
+    )
+
+
+def _guarded_connect(self, address):
+    if not _is_local(address):
+        _blocked(address)
+    return _real_connect(self, address)
+
+
+def _guarded_connect_ex(self, address):
+    if not _is_local(address):
+        _blocked(address)
+    return _real_connect_ex(self, address)
+
+
+def _guarded_create_connection(address, *args, **kwargs):
+    if not _is_local(address):
+        _blocked(address)
+    return _real_create_connection(address, *args, **kwargs)
+
+
+socket.socket.connect = _guarded_connect
+socket.socket.connect_ex = _guarded_connect_ex
+socket.create_connection = _guarded_create_connection
+
+
+# --------------------------------------------------------------------------- #
+# Gradio's mocked-OAuth path — the one legitimate import-time connection, removed
+# --------------------------------------------------------------------------- #
+# `gradio_ui` builds its `gr.Blocks` graph at module scope (that module-level `demo` is what HF
+# Spaces launches, so it stays that way). Off a Space, gradio's `attach_oauth` falls back to
+# `_add_mocked_oauth_routes`, which calls huggingface_hub `whoami` to mock a local profile.
+# That is gradio debug convenience, not our code, and it does not happen on the deployed Space
+# (there `SPACE_ID` is set and the real OAuth routes are used instead). It also requires the
+# developer's machine to be HF-logged-in, which CI is not.
+#
+# So we stub the mock. Patching `_get_mocked_oauth_info` — rather than `whoami` — is the
+# narrowest seam: it is the only function on that path that touches the network, and replacing
+# it leaves every route gradio registers intact, so the UI tests still exercise the real graph.
+# Nothing here changes the deployed app; conftest is test-only.
+def _install_offline_oauth_mock() -> None:
+    import gradio.oauth
+
+    gradio.oauth._get_mocked_oauth_info = lambda: {
+        "access_token": "test-token",
+        "token_type": "bearer",
+        "expires_at": 0,
+        "expires_in": 28800,
+        "scope": "openid profile",
+        "userinfo": {
+            "sub": "test-user-id",
+            "name": "Test User",
+            "preferred_username": "test-user",
+            "profile": "https://huggingface.co/test-user",
+            "picture": None,
+            "email": None,
+        },
+    }
+
+
+_install_offline_oauth_mock()
+
+
 @pytest.fixture
 def mini():
     return json.loads(_FIXTURE.read_text())

tests/test_hermetic.py

@@ -0,0 +1,69 @@
+"""The suite is hermetic, and the guard that makes it so is itself under test.
+
+`tests/conftest.py` claimed "NO live network" in a docstring for weeks while every
+fixture-using test called cbioportal.org, because curation fetched `study_metadata` and
+nothing patched it. The visible symptom was a run that failed with 12 errors and passed on
+retry — a third party's latency arriving as our bug, at whatever moment they were slow.
+
+The fix for *that* hole was to patch the function. The fix for the *class* of hole is this
+file: the socket guard is enforcement, and enforcement nobody tests decays into the same
+docstring. So these tests assert the guard is installed, that it actually refuses, and that
+it names the host it refused — because a guard that fires with an unhelpful message gets
+disabled by the next person who trips it.
+"""
+
+from __future__ import annotations
+
+import socket
+
+import pytest
+
+import conftest
+
+
+def test_socket_guard_is_installed():
+    """If someone removes the guard, this is the test that notices."""
+    assert socket.socket.connect is conftest._guarded_connect
+    assert socket.create_connection is conftest._guarded_create_connection
+
+
+def test_outbound_connection_is_refused():
+    """The guard must refuse a routable address — no DNS, no packets, no third party."""
+    with pytest.raises(AssertionError) as exc:
+        socket.create_connection(("www.cbioportal.org", 443), timeout=1)
+    assert "cbioportal" in str(exc.value)
+
+
+def test_refusal_names_the_host_and_says_what_to_do():
+    """A guard is only as good as the message it fails with."""
+    with pytest.raises(AssertionError) as exc:
+        socket.socket().connect(("huggingface.co", 443))
+    message = str(exc.value)
+    assert "huggingface.co" in message
+    assert "conftest" in message  # points at the fix, not just the symptom
+
+
+def test_loopback_is_allowed():
+    """Local sockets are not what this guard is about; blocking them would break test servers.
+
+    Asserted via `connect_ex` to a port nothing is listening on: a refused connection proves
+    the guard let the call through to the OS, which is the property under test.
+    """
+    with socket.socket() as s:
+        s.settimeout(1)
+        assert s.connect_ex(("127.0.0.1", 1)) != 0  # reached the OS and was refused there
+
+
+@pytest.mark.parametrize(
+    "module", ["test_ui_handler_gating", "test_deploy_surface", "test_machine_api_auth"]
+)
+def test_ui_modules_import_without_network(module):
+    """These three import `gradio_ui`, which builds its Blocks graph at module scope.
+
+    Off a Space, gradio's `attach_oauth` mocks the login button by calling HF `whoami` — a
+    real request, at import, before any test runs. conftest stubs that mock out. If the stub
+    stops matching gradio's internals (an upgrade renames `_get_mocked_oauth_info`), these
+    modules start hitting the network at *collection* time, which is exactly the failure this
+    suite is supposed to have stopped having. Importing them here proves the seam still holds.
+    """
+    __import__(module)