docs/architecture/02-deployment-topology.md

02 — Deployment Topology

Parent: Architecture hub · Sibling: 01 System overview · Next: 03 Request lifecycle

Status tags used in this document: IMPLEMENTED · VERIFIED · MEASURED · ATTEMPTED · NOT RUN · BLOCKED · DEFERRED · REJECTED · OPEN · RESOLVED · CLOSED.

One-paragraph summary. SatQuery AI is deployed as four tiers: a static browser client on Cloudflare Pages (satquery.pages.dev), a thin stateless gateway/orchestrator on Render (satquery-orchestrator at <backend-host>), a FastAPI inference service inside a GitHub Codespace (FastAPI on port 8000), and Hugging Face as the project/model-presence tier. The gateway does not dial into the Codespace. Instead the Codespace dials out to the gateway over a long-poll tunnel (POST /tunnel/agent), because a forwarded Codespace port returns HTTP 302 for a private repository. That inversion is the single most consequential decision in the topology, and it is why the deployment works at all with private repositories.

---

1. The four tiers

1.1 Tier map

USER / BROWSER
  │  HTTPS
  ▼
Cloudflare Pages — static frontend          satquery.pages.dev
  │  HTTPS, JSON, /api/*
  ▼
Render — orchestrator / API gateway         <backend-host>
  │  service: satquery-orchestrator
  │  outbound long-poll  (POST /tunnel/agent)   ← direction is INVERTED
  ▼
GitHub Codespace — FastAPI inference        potential-space-trout-r4ppw969w45j2pvvw :8000
  │  build_space_app(): /v1/health · /v1/capabilities · /v1/analyze · /v1/assets
  │  specialists: SmolVLM · RemoteCLIP · MiniLM · CROMA · STANet
  ▼
Hugging Face — project card + pinned model references

Sources: docs/DEPLOYMENT_TOPOLOGY.md §1; docs/FINAL_DELIVERY_TODO.md §1.2 (the same ASCII topology reproduced in the delivery single-source-of-truth); docs/FINAL_DELIVERY_REPORT.md §2.

flowchart LR
  U["Browser<br/>satquery.pages.dev"] -->|"HTTPS"| CF["Cloudflare Pages<br/>static frontend"]
  CF -->|"HTTPS JSON /api/*"| R["Render<br/>satquery-orchestrator"]
  R -->|"POST /tunnel/agent<br/>long-poll (outbound)"| A["Codespace tunnel agent"]
  A -->|"http://127.0.0.1:8000"| I["FastAPI<br/>build_space_app()"]
  I --> S[("SmolVLM · RemoteCLIP<br/>MiniLM · CROMA · STANet")]
  I -.->|"model refs"| HF["Hugging Face<br/>project + pinned models"]
  R -.->|"HF proxy path<br/>(not used in live config)"| HF

1.2 Tier responsibilities, at a glance

Tier	Host / identity	Runs	Holds secrets?	Holds state?
Browser	the user's machine	frontend/ static JS	No — never	no
Static	Cloudflare Pages, satquery.pages.dev	HTML/CSS/JS only	No	no
Gateway	Render, satquery-orchestrator, <backend-host>	deploy/render/main.py (SatQuery-Backend in production)	Yes — GITHUB_TOKEN (and HF_TOKEN if the proxy path is used)	no
Inference	GitHub Codespace potential-space-trout-r4ppw969w45j2pvvw, port 8000	app/space_app.py::build_space_app() via deploy/codespace/serve.py	no gateway secrets	ephemeral asset store only
Presence	Hugging Face (hf/)	project README / model cards	no	no

Sources: docs/DEPLOYMENT_TOPOLOGY.md §3.1–§3.4; render.yaml; deploy/render/main.py; app/space_app.py; .devcontainer/devcontainer.json.

1.3 Why exactly four tiers and not three

The plan forbids "unnecessary microservices" (docs/DEPLOYMENT_ARCHITECTURE.md §1.1, §6, quoting plan §73/§74). A gateway is nevertheless present, and docs/DEPLOYMENT_ARCHITECTURE.md §1.1 gives three concrete reasons rather than an architectural preference:

1. The inference host cannot hold the security boundary. It is a public ASGI app on third-party infrastructure. Rate limiting, size caps, CORS and secret custody belong outside it (docs/DEPLOYMENT_ARCHITECTURE.md §1.1 item 1).
2. A request that can be rejected on shape must never reach inference. In the original design the scarce resource was the ZeroGPU 5 GPU-minutes/day budget; in the active design it is inference wall time on a CPU Codespace. Either way the gateway is where a malformed request dies cheaply (docs/DEPLOYMENT_ARCHITECTURE.md §1.1 item 2).
3. The plan's §74 boundary excludes auth, multi-tenancy and queues. So the gateway is a proxy with validation, and must not grow into a platform (docs/DEPLOYMENT_ARCHITECTURE.md §1.1 item 3, §2.2).

The conclusion recorded in the source document is "two services, not three" — a static client, a gateway, and one inference service (docs/DEPLOYMENT_ARCHITECTURE.md §1.1). Hugging Face is a presence tier, not a runtime tier, in the active design.

---

2. Why a gateway exists

This section is the load-bearing one. A reader who understands only one part of the deployment should understand this: the gateway is not there to compute anything. It is there to be the boundary.

2.1 The responsibility table (authoritative)

docs/DEPLOYMENT_ARCHITECTURE.md §2.1 is the authoritative statement. Reproduced with the active host name substituted (Railway → Render):

Responsibility	Detail	Why it must be here
Schema validation	reject malformed bodies with the §5 error envelope	avoids spending inference on a request that will fail
Size limits	per-request body cap and per-file cap	the inference host cannot refuse a body it has already received
Rate limiting	per-IP count + window	back-pressure against accidental loops; fairness, not security — see §2.4
CORS	explicit allowlist of the frontend origin	never *
Request IDs	generate, inject, echo X-Request-Id	correlation across two services
Timeouts	upstream timeout shorter than the inference host's own budget	prevents a hung proxy holding a connection
Secret custody	GITHUB_TOKEN (and HF_TOKEN if used) live here only	the browser never sees them
Error translation	inference errors → the documented envelope	the error contract is a gateway product
Body relaying for upload	read and forward the raw body for POST /v1/assets	the upload path is not JSON-shaped, so JSON-oriented handling does not apply

2.2 The CORS allowlist is explicit, and never a wildcard

The orchestrator's CORS list is assembled by _allowed_origins() in deploy/render/main.py, in a documented order:

1. SATQUERY_ALLOWED_ORIGINS — the operator's comma-separated list. The authoritative source for any additional deployment origin.
2. _PRODUCTION_ORIGINS — ("https://satquery.pages.dev",), always present, so a deployment that forgets the environment variable still serves the real frontend. deploy/render/main.py records the reasoning: "an empty allowlist would otherwise take the live site down, which is a worse failure than the one this guards."
3. _DEV_ORIGINS — 20 enumerated host:port pairs (10 ports × localhost/127.0.0.1), added unless SATQUERY_ALLOW_DEV_ORIGINS is set to 0/false/no/"".

The dev-origin list is enumerated, not a regex and not a suffix match (deploy/render/main.py):

_DEV_ORIGINS: tuple[str, ...] = tuple(
    f"http://{host}:{port}"
    for host in ("localhost", "127.0.0.1")
    for port in ("3000", "5500", "5173", "8000", "8080")
)

A wildcard is refused in two places, deliberately:

• _allowed_origins() raises ValueError if "*" appears in the assembled list, and its docstring records why the check exists there as well as in the config validator: "this function cannot be the way a * reaches CORSMiddleware, which does not run that validator."
• GatewayConfig.__post_init__ (gateway/policy.py) refuses a wildcard at construction, so a misconfiguration fails at startup rather than on the first request.

allow_credentials=False is set explicitly in create_app() (deploy/render/main.py), matching the contract's "no auth, no cookies" position (docs/API_CONTRACT.md §7; plan §74).

The gateway also strips CORS headers coming back from upstream, so the CORS answer is the gateway's alone. gateway/app.py::_proxy asserts this rather than trusting it:

assert not any(_is_cors_header(k) for k in out_headers) or decision.headers, (
    "a CORS header reached the response without a policy decision; the "
    "upstream's headers are no longer filtered (see F-2)"
)

2.3 Size limits: two caps, both enforced twice, on purpose

Two independent caps exist, and they are different numbers with different jobs (docs/DEPLOYMENT_ARCHITECTURE.md §4):

Cap	Default	Scope	Where read
SATQUERY_MAX_FILE_BYTES	4 * 1024 * 1024 = 4,194,304 bytes	one uploaded file	both layers, from one variable
SATQUERY_MAX_BODY_BYTES	8 * 1024 * 1024	the whole request body	gateway

gateway/policy.py:221 declares max_file_bytes: int = 4 * 1024 * 1024; app/space_app.py::_asset_max_file_bytes() returns 4 * 1024 * 1024 when the variable is unset. The per-file cap is deliberately shared so the two layers cannot disagree about what "too large" means (docs/DEPLOYMENT_ARCHITECTURE.md §4).

SATQUERY_MAX_BODY_BYTES is enforced twice — from the Content-Length header and while reading the bytes — because the header check is declarative: it measures what the client claims. The measured consequence is in docs/DEPLOYMENT_ARCHITECTURE.md §4 (F-6), with the cap at 8 MiB and a 12 MiB body:

Client behaviour	Result	Peak allocation	Bytes read
Content-Length declared, 12 MiB	413 oversized_image	0.2 MiB	0
Content-Length omitted, 12 MiB	502 model_unavailable	13.9 MiB	12 MiB

and allocation tracked body size exactly with no ceiling: 1/8/16/32/64 MiB in → 3.0/8.1/16.0/32.0/64.0 MiB allocated. The remedy was to make the cap unconditional by enforcing it while reading, in the single shared reader gateway/assets.py::read_body_bounded, called by both layers (gateway/app.py::_read_body_bounded is now a thin adapter over it; app/space_app.py's /v1/assets handler calls the same function — that is F-9, which found the Space calling await request.body() and holding 64 MiB in → 128 MiB peak).

The honest framing, quoted from the source: "Operators should not treat the header check as the protection — it protects the gateway's memory against honest clients, not against hostile ones." (docs/DEPLOYMENT_ARCHITECTURE.md §4, F-6 note.)

2.4 Rate limiting is FAIRNESS, not security

This is a ruling, not an implementation detail. docs/DEPLOYMENT_ARCHITECTURE.md §5.2 carries the owner ruling of 2026-09-23:

"✅ RULED 2026-09-23 (owner ruling): the limiter is RETAINED as a fairness / rate-control mechanism only, and it is explicitly NOT a security or abuse-prevention boundary."

The measurement that forced the ruling is reproduced here because it is the whole argument. Limit set to 3 requests / 60 s, 8 requests sent in-process:

Case	Statuses	Throttled
One client, no X-Forwarded-For	502 502 502 429 429 429 429 429	5 / 8
A fresh spoofed X-Forwarded-For per request	502 502 502 502 502 502 502 502	0 / 8

The mechanism is gateway/app.py::_client_ip, which derives the rate-limit key from the first hop of X-Forwarded-For — a client-supplied header. Its own docstring already said the value is attacker-controlled and is "a rate-limit key, not an identity"; what the measurement added is that the limiter does not hold at all against a caller willing to vary one header.

Consequences a deployment must honour (docs/DEPLOYMENT_ARCHITECTURE.md §5.2):

• Do not size abuse protection on this limiter. It is not that control.
• A 429 is a fairness signal, not a security signal, and its absence is not evidence that no abuse occurred.
• The gateway remains the request-side boundary for shape, size and content type — the things it can actually enforce. Rate is not one of them.

The limit itself is two variables because the limit is the pair (docs/DEPLOYMENT_ARCHITECTURE.md §4): SATQUERY_RATE_LIMIT_PER_IP and SATQUERY_RATE_LIMIT_WINDOW_S; 10 and 60.0 mean "ten per minute".

Why there is no code fix. Correctly trusting X-Forwarded-For requires knowing how many proxy hops the platform inserts — a deployment fact not verifiable from the build host. Hard-coding an assumption would replace a documented weakness with an undocumented one (docs/DEPLOYMENT_ARCHITECTURE.md §5.2).

2.5 Request IDs

The gateway generates a request id, injects it on the upstream leg, and echoes it to the client (gateway/app.py::_proxy):

headers = policy.upstream_headers(dict(request.headers), token=token)
headers["X-Request-Id"] = decision.request_id

Every non-2xx envelope the gateway owns carries the same id, including ones raised by the framework's own 404/405 handler, which is registered explicitly (gateway/app.py):

@app.exception_handler(StarletteHTTPException)
async def _contract_envelope_for_transport_errors(request, exc):
    code = "routing_error" if exc.status_code < 500 else "satquery_error"
    status, body = translate_error(...)

The comment above that handler records the measurement that motivated it: before the fix, GET /v1/whocares → 404 {"detail":"Not Found"} and GET /v1/assets → 405 {"detail":"Method Not Allowed"}, while every handler-owned path answered with the contract envelope. A client written to the contract parses error.code and would get a KeyError exactly when it is trying to explain a failure to a user. app/space_app.py carries the same handler for the same reason (F-12/F-12b) — it was previously registered on the gateway only.

2.6 Timeouts, and the no-retry rule

Timeout	Default	Meaning
SATQUERY_UPSTREAM_TIMEOUT_S	90.0	gateway → inference request timeout; must sit inside the task budget
SATQUERY_WAKE_TIMEOUT_S	120	how long the gateway polls for readiness before giving up
SATQUERY_TUNNEL_TIMEOUT_S	150	how long a tunnel request parks before returning tunnel_offline

Defaults are declared in deploy/render/main.py:

def _wake_timeout_s() -> float:
    return float(os.environ.get("SATQUERY_WAKE_TIMEOUT_S", "120"))

def _upstream_timeout_s() -> float:
    return float(os.environ.get("SATQUERY_UPSTREAM_TIMEOUT_S", "90"))

The gateway never retries POST /v1/analyze. gateway/app.py::_proxy states it inline:

except Exception as exc:  # network-level failure
    # NO RETRY. A retry on /v1/analyze would spend GPU quota twice
    # (docs/DEPLOYMENT_ARCHITECTURE.md section 2.2).

and docs/DEPLOYMENT_TOPOLOGY.md §2 repeats it for the active design: "Render must not retry POST /api/infer on its own — a retry would consume inference a second time. The client decides on retry." The client-side consequence is a hard rule in the frontend contract: never automatically retry POST /v1/analyze (docs/FRONTEND_INTEGRATION.md §6.1).

2.7 Secret custody

Secret	Lives	Never
GITHUB_TOKEN	Render environment only	in the browser, in the repo, in a client bundle
HF_TOKEN	Render environment only, if the HF proxy path is used	as above

The live Render configuration was measured on 2026-09-25 and has no SATQUERY_UPSTREAM_URL and no HF_TOKEN (docs/DEPLOYMENT_TOPOLOGY.md header note; release/repo/docs/DEPLOYMENT.md §3.1). The token that is present is GITHUB_TOKEN — needed only by the GitHub-API wake path, and reported in the health payload as a boolean, never a value:

"has_github_token": bool(os.environ.get("GITHUB_TOKEN")),

docs/FRONTEND_INTEGRATION.md §7 states the frontend requirement plainly: no secrets in the browser, talk only to the gateway, and never call the inference host directly — "it is not the security boundary and its CORS will not welcome you."

This document contains no credential, token, key or password, and no path to a credential file. Every secret is described by where it lives, never by its value.

2.8 Error translation, and one rule about codes

The gateway translates upstream failures into the documented envelope but passes the code through unchanged (docs/DEPLOYMENT_ARCHITECTURE.md §2.3):

"The code is passed through unchanged. The gateway must not invent codes: the taxonomy in core/errors.py is the single source of truth, and a gateway that remapped it would make the frontend's error handling unpredictable."

The envelope shape is fixed (docs/DEPLOYMENT_ARCHITECTURE.md §2.3):

{
  "error": {
    "code": "pair_misaligned",
    "message": "The images are not sufficiently co-registered for spatial analysis.",
    "detail": "RMSE 4.21 px exceeds the 2.0 px budget",
    "recoverable": false,
    "request_id": "req_01H...",
    "run_id": "9f2c1c0e-..."
  }
}

The orchestrator's own translation table is small and explicit (deploy/render/main.py):

Orchestrator error class	code	HTTP	recoverable
WakeTimeout	wake_timeout	504	true
OrchestratorConfigError	orchestrator_config_error	500	false
OrchestratorUpstreamError	upstream_unreachable	502	true
connection/timeout to upstream (_proxy)	upstream_unreachable	502	true
other transport error (_proxy)	upstream_error	502	true
non-JSON upstream body (_proxy)	schema_validation_error	502	true
non-JSON request body (/api/infer)	invalid_request	400	false

A non-JSON upstream body is a defect, not a pass-through. gateway/app.py::_proxy enforces this for every status, not only 2xx, and the comment records why: the guard originally read upstream.status_code < 400, so a non-JSON 4xx/5xx — a proxy error page, an HTML 502 from a load balancer, a plain-text stack trace — was forwarded verbatim. A sandbox egress proxy returned a 502 whose body disclosed os error 10061; that is how it was found. /v1/health is exempt because a liveness probe may legitimately answer non-JSON.

A transport failure's raw exception text is never published (F-15c, owner ruling 2026-09-23). gateway/app.py::_transport_failure_detail maps the exception's MRO class names to a path-free classification:

_TRANSPORT_FAILURES: tuple[tuple[str, str], ...] = (
    ("TimeoutException", "the upstream did not respond within the gateway timeout"),
    ("ConnectError", "the upstream could not be reached"),
    ("ProxyError", "the gateway's egress proxy refused the connection"),
)

The full exception still reaches the operator through _log.error(..., exc_info=exc). It is moved, not deleted.

2.9 What the gateway must NOT do

docs/DEPLOYMENT_ARCHITECTURE.md §2.2 is a closed list:

• No persistence. No database, no Redis, no session store.
• No model inference.
• No auth system (plan §74).
• No request queue (plan §73 forbids Redis-cluster/queue infrastructure).
• No retries on POST /v1/analyze.
• No second copy of the capability table. The gateway proxies /v1/capabilities and nothing else decides that question. The authoritative sources for asset counts are core.planner.CAPABILITY_ASSETS and SpecialistSpec.requires_assets; per _indices_for's docstring, "duplicating that logic here would give two places to disagree."
• No asset storage. The gateway relays upload bytes; it does not retain them. The store lives with the inference host, which is the only component that will read them back (app/space_app.py::get_asset_store docstring).

deploy/render/main.py's module docstring states the same three absences in one line: "It holds no model, no state, no database, and performs no auth (per plan §73/§74)." And it repeats the capability-table rule: "There is deliberately no second copy of the capability table here; the gateway proxies /v1/capabilities and nothing else decides that question."

2.10 The proxied route allowlist

The gateway forwards an allowlist, not a passthrough (gateway/app.py):

PROXIED_ROUTES: tuple[str, ...] = (
    "/v1/health",
    "/v1/capabilities",
    "/v1/analyze",
    "/v1/assets",
)

COSTLY_ROUTES: tuple[str, ...] = ("/v1/analyze", "/v1/assets")

The two tuples answer different questions and are deliberately separate — "may this reach the Space at all?" versus "does it cost a metered resource?" — because collapsing them would make the rate limiter's coverage depend on the proxy allowlist (gateway/app.py).

BLOCKED_ROUTES is empty, and the comment says it should stay that way: the tuple exists so a route the contract discusses but the server does not implement answers 501 with a reason instead of a 404 a frontend developer would debug as a typo.

On the orchestrator side the four routes are /api/health, /api/infer, /api/capabilities, /api/assets, each proxying to the matching /v1/* route (docs/DEPLOYMENT_TOPOLOGY.md §3.2; deploy/render/main.py). /api/health is the exception: it never answers for the inference host. Its docstring says so — "Reports its own configuration; never answers for the Codespace (that is /api/capabilities)."

---

3. Why the transport is an outbound tunnel

3.1 The forwarded-port failure

A GitHub Codespace exposes a forwarded port publicly, but for a private repository that forwarded URL returns HTTP 302 — a redirect to a sign-in page, not the service. docs/DEPLOYMENT_TOPOLOGY.md records this in its measured note:

"Transport is an outbound tunnel, not a polled forwarded port: the Codespace runs deploy/codespace/tunnel_agent.py, which dials out to POST /tunnel/agent (long-poll) and executes against http://127.0.0.1:8000 locally."

release/repo/docs/DEPLOYMENT.md §7 lists it among the platform traps:

"A forwarded Codespace port returns 302 for a private repo — which is why the tunnel exists."

and docs/DEPLOYMENT_DECISION.md's correction banner records the historical position and its reversal:

"Codespaces were not dropped; the forwarded-port path is dead (HTTP 302 for a private repo) and an outbound tunnel is used instead."

3.2 What the inversion buys

deploy/codespace/launch.sh states the property in its header comment, and it is worth quoting because it is the whole reason the design is robust to repository visibility:

"The tunnel is why this works with a PRIVATE repository: the agent makes only outbound HTTPS calls, so GitHub's port-forwarding relay, port visibility and the repository's visibility are all irrelevant. The orchestrator never dials into this Codespace."

Consequences, each observable:

Property	Value under the tunnel
Repository visibility	irrelevant — only outbound HTTPS is used
Port visibility setting	irrelevant
Inbound firewall / NAT	no inbound connection is required at all
Who initiates	the Codespace, to SATQUERY_HUB_URL
What the hub needs	a long-poll endpoint and a way to match a response to a pending request

3.3 Direction, restated as a diagram

sequenceDiagram
  autonumber
  participant CF as "Cloudflare Pages"
  participant R as "Render hub"
  participant TA as "Codespace tunnel agent"
  participant API as "FastAPI :8000"

  Note over TA,R: startup — agent dials OUT
  TA->>R: POST /tunnel/agent (announce, long-poll)
  R-->>TA: (holds the poll open)

  CF->>R: POST /api/infer
  R->>TA: deliver request on the open poll
  TA->>API: POST http://127.0.0.1:8000/v1/analyze
  API-->>TA: ResultEnvelope
  TA-->>R: response
  R-->>CF: envelope + X-SatQuery-State

Honest note on the agent's internals. deploy/codespace/launch.sh invokes python deploy/codespace/tunnel_agent.py and greps its log for the string announced to hub. That file is not present in the monorepo working tree and is not tracked by git (see §9.4), so its function names, arguments and payload shapes are UNKNOWN — not established from the available evidence. What is established is: the agent exists in the production SatQuery-Inference repository (docs/FINAL_DELIVERY_TODO.md §1.3), it dials SATQUERY_HUB_URL, it executes against http://127.0.0.1:8000, and it is supervised by deploy/codespace/launch.sh.

3.4 The observable proof of transport

The frontend treats a response header as the evidence that the hub forwarded to the Codespace rather than answering locally. frontend/assets/js/live.js reads it, and the unit suite pins the read:

"x-satquery-transport: tunnel is the proof that Render forwarded to the Codespace rather than answering locally. It is only readable before the response object is discarded." (tests/unit/test_frontend_live_wiring.py, test_the_client_reads_the_transport_header_as_evidence)

The measured live value is x-satquery-transport: tunnel on POST /api/infer (docs/FINAL_DELIVERY_TODO.md §1.4, §6 E-03; docs/FINAL_DELIVERY_REPORT.md §3 P3).

---

4. Wake flow

4.1 The flow

The inference Codespace is CPU-first and may be stopped when idle. Before a request can be served the hub starts it (if stopped) and polls health until it answers. The frontend shows "Waking inference engine…" while this happens (docs/DEPLOYMENT_TOPOLOGY.md §2).

sequenceDiagram
  participant CF as "Cloudflare Pages"
  participant R as "Render hub"
  participant C as "GitHub Codespace"
  participant HF as "Hugging Face"

  CF->>R: GET /api/health (or POST /api/infer)
  R->>C: is the Codespace running?
  alt stopped
    R->>C: start Codespace
    R->>C: poll GET /v1/health
    C-->>R: 200 {status: ok|degraded}
    R-->>CF: "Waking inference engine…"
  end
  CF->>R: POST /api/infer (query + assets)
  R->>C: POST /v1/analyze
  C->>HF: resolve pinned model references
  C-->>R: ResultEnvelope
  R-->>CF: result (envelope + error translation)

Source: docs/DEPLOYMENT_TOPOLOGY.md §2 (verbatim structure).

4.2 The wake path in code

deploy/render/main.py::ensure_codespace_up() is the wake implementation. Its contract is precise:

async def ensure_codespace_up() -> tuple[str, bool]:
    """Ensure the Codespace is running; return ``(base_url, woke)``.

    Steps:
      1. ``GET`` the Codespace via the GitHub API.
      2. If ``state != "available"``, ``POST .../start``.
      3. Poll ``GET {base}/v1/health`` until 200 or until
         ``SATQUERY_WAKE_TIMEOUT_S`` elapses.
    """

Its polling knobs are module constants:

_WAKE_POLL_INTERVAL_S = 2.0
_WAKE_HEALTH_TIMEOUT_S = 10.0

and the failure mapping is explicit: a GitHub auth/transport failure becomes OrchestratorUpstreamError (502, recoverable), a missing Codespace name becomes OrchestratorConfigError (500, not recoverable), and an exhausted deadline raises WakeTimeout (504, recoverable) with the last probe error in the detail.

4.3 The response header the client reads

/api/infer tags the proxied response so the frontend can tell whether the delay was a cold start (deploy/render/main.py):

out = await _proxy("POST", f"{base}/v1/analyze", json=body)
out.headers["X-SatQuery-State"] = "waking" if woke else "ready"
return out

The unit suite pins both headers on the client side: assert "x-satquery-transport" in source and assert "X-SatQuery-State" in source (tests/unit/test_frontend_live_wiring.py).

4.4 The wake path is a fallback in the tunnel design

The measured note in docs/DEPLOYMENT_TOPOLOGY.md §2 is explicit that the tunnel design does not depend on the GitHub-API wake:

"The GitHub-API wake path (POST /user/codespaces/{name}/start) still exists but the tunnel design relies on the agent reconnecting on Codespace start via the devcontainer postStartCommand."

So there are two mechanisms and they are not equivalent:

Mechanism	Trigger	Effect when it works	Effect when it fails
Devcontainer postStartCommand → launch.sh → tunnel agent	every Codespace start	agent reconnects; agent_connected: true	agent_connected: false; /api/infer parks to SATQUERY_TUNNEL_TIMEOUT_S
GitHub-API wake (ensure_codespace_up)	any /api/* request	starts a stopped Codespace, polls /v1/health	wake_timeout (504, recoverable)

---

5. Cold start — documented, not hidden

Render's free tier sleeps when idle, and the Codespace may be stopped (the live GitHub value recorded is idle_timeout_minutes=30, docs/FINAL_DELIVERY_TODO.md §6 E-04). The measured statement is:

"Render's free tier also sleeps when idle. Cold start is therefore tens of seconds and is documented, not hidden." (docs/DEPLOYMENT_TOPOLOGY.md §2)

The UI consequence is recorded in docs/FRONTEND_INTEGRATION.md §6:

Constraint	Value	UI consequence
Cold start	tens of seconds	"A determinate-looking progress bar would lie. Use an indeterminate state with a 'this can take up to a minute' hint."

and the operator consequence in docs/FINAL_DELIVERY_REPORT.md §8:

"Warm the demo stack ~10 min before presenting: open the Codespace and confirm GET /api/health shows tunnel.agent_connected:true. If the Codespace idle-stops, restart it (the tunnel agent reconnects via the devcontainer postStartCommand)."

No latency characterisation exists. docs/FRONTEND_INTEGRATION.md §9 states it plainly: "Latency is not characterized. No cold-start or throughput measurement has been taken against a live Space." The phrase "tens of seconds" is a documented expectation, not a measurement. A precise cold-start distribution is UNKNOWN — not established from the available evidence.

---

6. transport_mode: auto, the fallthrough, and B-07

6.1 The live transport configuration

The live Render service reports its transport settings in the health payload. Measured 2026-09-25:

Setting	Live value
transport_mode	auto
tunnel_timeout_s	150.0
wake_timeout_s	120.0
upstream_timeout_s	90.0

Source: release/repo/docs/DEPLOYMENT.md §2 (live payload) and docs/DEPLOYMENT_TOPOLOGY.md header note.

6.2 The fallthrough, exactly

docs/FINAL_DELIVERY_TODO.md §5 (blocker register, row B-07) records the confirmed root shape:

"Root shape confirmed 2026-09-25: in auto transport mode a tunnel timeout falls through to the forward path (SatQuery-Backend/main.py:546), which then burns wake_timeout_s=120 on a 302 → the observed 504."

DELIVERY_REPORT_2026-09-25.md §4 gives the mechanism and the arithmetic:

"in auto transport mode a tunnel timeout falls through to the forward path (main.py:546 returns early only when mode == "tunnel"); the forward path then burns wake_timeout_s = 120 on a 302. Measured timing ≈ 249 s ≈ tunnel_timeout_s=150 + wake_timeout_s=120."

So the worst case is:

tunnel park        150 s   (SATQUERY_TUNNEL_TIMEOUT_S)
   + wake poll     120 s   (SATQUERY_WAKE_TIMEOUT_S)
   -------------------------
   ≈ 249 s  → a 504 the client waited four minutes for

flowchart TD
  A["POST /api/infer<br/>transport_mode = auto"] --> B{"tunnel agent<br/>connected?"}
  B -- yes --> C["execute via tunnel<br/>x-satquery-transport: tunnel"]
  B -- "no / timeout" --> D["tunnel park expires<br/>SATQUERY_TUNNEL_TIMEOUT_S = 150 s"]
  D --> E{"mode == tunnel?"}
  E -- yes --> F["return tunnel_offline<br/>503 recoverable"]
  E -- "no (auto) → FALLS THROUGH" --> G["forward path:<br/>forwarded port answers 302"]
  G --> H["burns wake_timeout_s = 120 s<br/>polling health"]
  H --> I["wake_timeout<br/>504 recoverable"]
  style I fill:#fde,stroke:#c33
  style D fill:#ffe,stroke:#cc3

6.3 B-07 is OPEN

B-07 — Transient tunnel-agent gaps — is OPEN. Stated three times in the sources so it cannot be mistaken:

"B-07 | Transient tunnel-agent gaps | OPEN | A request can hang or return 504 (tunnel_offline / wake timeout; the forwarded port returns 302). Observed once live. Mitigation: keep the Codespace warm before the demo; the client shows an actionable retry message." (docs/FINAL_DELIVERY_REPORT.md §6)

*"B-07 | Transient tunnel-agent gaps (agent briefly absent) → a request can hang or return 504 (tunnel_offline / wake timeout, forward path 302) | … | OPEN — patch prepared, not deployed."* (docs/FINAL_DELIVERY_TODO.md §5)

"B-07 backend patch prepared, NOT deployed." (docs/FINAL_DELIVERY_TODO.md sprint-status note)

6.4 The prepared patch — prepared, NOT deployed

The patch is fix-b07-forward-unavailable.patch, in the session workspace at .workbuddy-ai/scratch/deployed-backend/fix-b07-forward-unavailable.patch (DELIVERY_REPORT_2026-09-25.md §8). Its content and verification:

Item	Detail
Base	the deployed SatQuery-Backend/main.py @ 89d80eaddec5 (769 lines)
Size	9 hunks plus a 340-line test
Change A	adds forward_unavailable (503, recoverable: true) for a terminal 302/401/403 on the forward path, instead of burning the wake timeout
Change B	adds upstream_timeout (504) for "tunnel healthy but slow"
Change C	fixes /api/health codespace_name trailing \n via .strip()
Independent verification	git apply --check clean, git apply clean, py_compile OK
Presence check	forward_unavailable @ main.py:326, upstream_timeout @ :601, codespace_name .strip() @ :686
Deployment status	NOT deployed

Sources: DELIVERY_REPORT_2026-09-25.md §4; docs/FINAL_DELIVERY_TODO.md §6 E-12.

A retracted claim, recorded because the honesty matters. The report records that an earlier claim that the patch "would not have prevented" the observed 504 "was wrong and was retracted". The corrected position: "Change A is genuinely on the failing path — it converts a 504-after-249 s into a 503-early with an actionable code." (DELIVERY_REPORT_2026-09-25.md §4.)

Why it is not deployed: "the patch is not needed for the demo and touches the live backend. The residual is better mitigated operationally (keep the Codespace warm, raise the idle timeout)." (DELIVERY_REPORT_2026-09-25.md §4.)

6.5 Operational trap recorded with the patch

"the local C:/Users/anish/SatQuery-Backend (680 lines) is STALE. Always fetch the deployed main.py before touching backend code." (DELIVERY_REPORT_2026-09-25.md §4)

This is the same class of trap as §9.4 below: the working copy is not the deployed source.

---

7. The full live health payload

7.1 The measured payload

Probed live on 2026-09-25 against https://<backend-host>/api/health (release/repo/docs/DEPLOYMENT.md §2):

{"status":"ok","service":"satquery-orchestrator",
 "tunnel":{"agent_connected":true,"agent_id":"codespaces-fd1038","pending":0,"completed":97},
 "config":{"codespace_name":"potential-space-trout-r4ppw969w45j2pvvw\n","codespace_port":8000,
           "transport_mode":"auto","tunnel_timeout_s":150.0,"wake_timeout_s":120.0,
           "upstream_timeout_s":90.0,"device":"cpu","has_github_token":true}}

Exact command used elsewhere in the project's evidence register: curl --noproxy '*' https://<backend-host>/api/health (docs/FINAL_DELIVERY_REPORT.md §4).

7.2 Field-by-field

Field	Type	Meaning	Live value
status	string	the hub's own liveness	"ok"
service	string	the service identity	"satquery-orchestrator"
tunnel.agent_connected	bool	is a tunnel agent currently polling?	true
tunnel.agent_id	string	which agent identity holds the poll	"codespaces-fd1038"
tunnel.pending	int	requests delivered but not yet answered	0
tunnel.completed	int	requests completed since the agent connected	97
config.codespace_name	string	the target Codespace	"…pvvw\n" — carries a trailing \n
config.codespace_port	int	the inference port	8000
config.transport_mode	string	transport selection	"auto"
config.tunnel_timeout_s	float	tunnel park budget	150.0
config.wake_timeout_s	float	wake poll budget	120.0
config.upstream_timeout_s	float	proxy request timeout	90.0
config.device	string	declared device	"cpu"
config.has_github_token	bool	is a GitHub token configured?	true

7.3 completed was observed at three different values — do not treat any as a constant

The tunnel counter is a monotonic runtime counter, not a fixed fact. Three measured readings exist, each with its own provenance:

Reading	Where recorded
completed: 97	release/repo/docs/DEPLOYMENT.md §2 (the live health probe)
completed: 314	docs/FINAL_DELIVERY_TODO.md §1.4 and §6 E-02; docs/DEPLOYMENT_TOPOLOGY.md §2
completed: 338	docs/FINAL_DELIVERY_REPORT.md §3 P2

They are consistent with each other — the counter grows — and the honest statement is "completed was measured at 97, 314 and 338 at three different times on 2026-09-25." Quoting any one of them as the value would be wrong.

7.4 codespace_name carries a trailing newline — B-02, OPEN (cosmetic)

config.codespace_name reports …pvvw\n. This is B-02, and its status is OPEN but cosmetic:

"P2-T03 /api/health codespace_name trailing \n | DEFERRED (cosmetic) | Wake path is safe (_codespace_name() strips, main.py:123,357); only the health payload reports the raw value." (docs/FINAL_DELIVERY_REPORT.md §6)

"B-02 | /api/health reports codespace_name with a trailing \n | Cosmetic — reporting only; the wake path strips via _codespace_name() (main.py:123,357) | P2-T03 | none needed | DOWNGRADED" (docs/FINAL_DELIVERY_TODO.md §5)

The fix is known and one line — "change line 619 to _codespace_name(), then Render redeploys" (docs/FINAL_DELIVERY_TODO.md §4, P2-T03) — and the row's own reasoning for deferring is that "a live-backend redeploy before the demo is not worth the risk."

Do not upgrade this. B-02 is OPEN. It is not RESOLVED, and it is not CLOSED.

7.5 The orchestrator's own /api/health shape in the repository

deploy/render/main.py — the monorepo copy, which is not the deployed source (§9.4) — declares a different, simpler health payload. Reproduced because it documents the contract of the route even where the deployed implementation has grown:

@app.get("/api/health")
async def health() -> dict[str, Any]:
    """Orchestrator liveness. Reports its own configuration; never answers
    for the Codespace (that is /api/capabilities)."""
    return {
        "status": "ok",
        "service": "satquery-orchestrator",
        "config": {
            "codespace_name": os.environ.get("CODESPACE_NAME", ""),
            "codespace_port": _codespace_port(),
            "has_github_token": bool(os.environ.get("GITHUB_TOKEN")),
            "allowed_origins": _allowed_origins(),
            "production_origins": list(_PRODUCTION_ORIGINS),
            "dev_origins_enabled": _dev_origins_enabled(),
            "wake_timeout_s": _wake_timeout_s(),
            "upstream_timeout_s": _upstream_timeout_s(),
            "device": os.environ.get("SATQUERY_DEVICE", ""),
        },
    }

Note the design decision visible here: allowed_origins reports the effective list, so an operator can confirm from outside what the service will actually accept — not just what they set. The comment says the dev entries being visible "is how a production deployment proves it turned them off."

The discrepancy is real and is stated rather than smoothed over. The deployed payload carries a tunnel block and config.transport_mode / config.tunnel_timeout_s, which the monorepo copy does not. The monorepo copy is a 532-line file with no tunnel code at all; the deployed SatQuery-Backend/main.py is 768–769 lines with it (docs/FINAL_DELIVERY_TODO.md §1.1; DELIVERY_REPORT_2026-09-25.md §4).

---

8. Environment variables

8.1 Render (orchestrator) — measured live values

Variable	Live value	Purpose
CODESPACE_NAME	potential-space-trout-r4ppw969w45j2pvvw	which Codespace to target
CODESPACE_PORT	8000	the inference port on that Codespace
SATQUERY_ALLOWED_ORIGINS	https://satquery.pages.dev	CORS allowlist (the Pages origin)
SATQUERY_DEVICE	cpu	declared device
SATQUERY_TRANSPORT	auto	transport selection
SATQUERY_TUNNEL_TIMEOUT_S	150	tunnel park budget
SATQUERY_WAKE_TIMEOUT_S	120	wake poll budget
SATQUERY_UPSTREAM_TIMEOUT_S	90	gateway → upstream request timeout
GITHUB_TOKEN	present	GitHub API wake path; never sent to the browser

Source: docs/DEPLOYMENT_TOPOLOGY.md header note (measured against GET /api/health); release/repo/docs/DEPLOYMENT.md §3.1.

Two absences are as important as the presences. There is no SATQUERY_UPSTREAM_URL and no HF_TOKEN in the live configuration (docs/DEPLOYMENT_TOPOLOGY.md; release/repo/docs/DEPLOYMENT.md §3.1). SATQUERY_UPSTREAM_URL is absent because the transport is the outbound tunnel, not a forwarded port; HF_TOKEN is absent because the HF proxy path is not used live.

8.2 Render — the blueprint's declared variables

render.yaml (the blueprint) declares the same vocabulary as a service definition:

services:
  - type: web
    name: satquery-orchestrator
    runtime: python
    plan: free
    buildCommand: pip install -r deploy/render/requirements.txt
    startCommand: uvicorn deploy.render.main:app --host 0.0.0.0 --port $PORT
    healthCheckPath: /api/health
    envVars:
      - key: PORT
        sync: false
      - key: SATQUERY_ALLOWED_ORIGINS
        sync: false
      - key: GITHUB_TOKEN
        sync: false
      - key: CODESPACE_NAME
        sync: false
      - key: CODESPACE_PORT
        value: "8000"
      - key: SATQUERY_DEVICE
        value: "cpu"
      - key: SATQUERY_WAKE_TIMEOUT_S
        value: "120"
      - key: SATQUERY_UPSTREAM_TIMEOUT_S
        value: "90"

Three things this file establishes that are easy to miss:

1. plan: free — the free tier, which is why Render sleeps when idle (§5).
2. healthCheckPath: /api/health — the platform's own liveness probe points at the orchestrator's self-report route, which never touches the inference host.
3. sync: false on SATQUERY_ALLOWED_ORIGINS, GITHUB_TOKEN, CODESPACE_NAME and PORT means those are operator-supplied, not blueprint-committed. No secret value appears in the repository.

The blueprint does not declare SATQUERY_TRANSPORT or SATQUERY_TUNNEL_TIMEOUT_S, which the live service reports. The blueprint and the live service have diverged. Whether the live service sets them through the dashboard or through a newer blueprint is UNKNOWN — not established from the available evidence; what is established is the live value set in §8.1.

8.3 Codespace (inference) — declared and effective

Variable	Where set	Purpose
PORT	containerEnv = "8000", re-exported by launch.sh	platform-assigned; must be read (historical blocker #2)
SATQUERY_DEVICE	containerEnv = "cpu", re-exported by launch.sh	cpu | cuda | mps | null; read without importing torch
SATQUERY_ASSET_ENABLED	containerEnv = "1", re-exported by launch.sh	enables POST /v1/assets; both this and the dir are required
SATQUERY_ASSET_DIR	containerEnv = "/tmp/satquery-assets", re-exported by launch.sh	where uploaded bytes are written
SATQUERY_MAX_FILE_BYTES	not set live (default applies)	per-file cap, shared with Render
SATQUERY_ASSET_MAX_FILES	not set live (default applies)	optional handle capacity, default 32
SATQUERY_ASSET_TTL_S	not set live (default applies)	optional handle lifetime, default 900.0
SATQUERY_HUB_URL	defaulted by launch.sh	the hub the agent dials
PYTHONPATH	set by launch.sh	repo root, so import app resolves

Sources: .devcontainer/devcontainer.json; deploy/codespace/launch.sh; app/space_app.py.

.devcontainer/devcontainer.json in full:

{
  "name": "SatQuery AI — Codespace Inference",
  "image": "mcr.microsoft.com/devcontainers/python:3.12",
  "forwardPorts": [8000],
  "portsAttributes": {
    "8000": { "label": "SatQuery inference", "visibility": "public" }
  },
  "containerEnv": {
    "SATQUERY_DEVICE": "cpu",
    "PORT": "8000",
    "SATQUERY_ASSET_ENABLED": "1",
    "SATQUERY_ASSET_DIR": "/tmp/satquery-assets"
  },
  "postCreateCommand": "bash deploy/codespace/post_create.sh",
  "postStartCommand": "bash deploy/codespace/launch.sh",
  "customizations": { "vscode": { "extensions": ["ms-python.python"] } }
}

A trap worth recording, from launch.sh's own comment: "containerEnv is only applied when the container is CREATED, so setting it there alone would leave an already-running Codespace unconfigured until a rebuild. This script runs on every start and is therefore the effective source of truth." The variables are therefore set twice — in containerEnv and in launch.sh — and launch.sh is the one that governs a running container.

8.4 The historical vocabulary — still the contract

docs/DEPLOYMENT_ARCHITECTURE.md §4 remains authoritative for the env-var vocabulary; only host names moved. Its full table, reproduced, with the active host substituted:

Variable	Where it lives (historical → active)	Purpose
HF_TOKEN	Railway only → Render only, if used	upstream credential; never sent to the browser
SATQUERY_SPACE_URL	Railway → SATQUERY_UPSTREAM_URL	upstream URL
SATQUERY_ALLOWED_ORIGINS	Railway → Render	CORS allowlist
PORT	Railway → Render	supplied by the platform
SATQUERY_DEVICE	Space → Codespace	cpu | cuda | mps | null; read without importing torch
SATQUERY_ASSET_ENABLED	Space → Codespace	enables POST /v1/assets; fails closed
SATQUERY_ASSET_DIR	Space → Codespace	where uploaded bytes are written
SATQUERY_MAX_FILE_BYTES	both	per-file size cap, read by both layers from one variable
SATQUERY_MAX_BODY_BYTES	Railway → Render	whole-request body cap, above the per-file cap
SATQUERY_UPSTREAM_TIMEOUT_S	Railway → Render	gateway → upstream timeout; default 90.0
SATQUERY_RATE_LIMIT_PER_IP / _WINDOW_S	Railway → Render	per-IP count + window
SATQUERY_ASSET_MAX_FILES / SATQUERY_ASSET_TTL_S	Space → Codespace	optional handle capacity / lifetime

Three notes from that section are worth carrying forward because they explain why the vocabulary has this shape:

1. SATQUERY_MAX_FILE_BYTES is applied while reading at both layers, not after (F-9, F-6). Both layers call the single reader gateway/assets.py::read_body_bounded, so the two enforcement points cannot drift.
2. Both layers refuse an unparsable or non-positive value and name the variable (F-7). Reading one variable is not the same as agreeing on its value: the two parsers previously diverged in opposite directions — 'abc' raised at the gateway but silently defaulted to 4 MiB on the inference host; '0' was accepted at the gateway but rejected on the inference host. A malformed cap now fails startup at both layers rather than running on a limit nobody chose.
3. None of the asset variables is a config key, and that is deliberate: adding a key to configs/base.yaml moves Config.hash off 78f1e3700da15aa1 and invalidates the frozen Phase-9 benchmark. Asset storage is deployment state, so it is read from the environment.

The F-8 note on SATQUERY_DEVICE is also load-bearing and is reproduced in §8.5.

8.5 SATQUERY_DEVICE: four read sites, and the case bug

docs/DEPLOYMENT_ARCHITECTURE.md §4 records that the served value is validated against the contract's closed set. Measured before the fix, SATQUERY_DEVICE had four read sites and only three normalised:

Site	Behaviour before the fix
core/config.py:88 (Config.device_preference)	raw — no strip, no lower
app/deployment.py:570 (gpu_available)	.strip().lower()
app/deployment.py:946 (the served device resolver)	.strip(), no lower — the odd one
app/deployment.py:970 (_cuda_detected)	.strip().lower()

Two defects followed, both measured:

• Case changed the answer. 'cuda' → 'cpu' but 'CUDA' → 'CUDA', so one payload could announce gpu_available: true alongside device: "CUDA" — a GPU is claimed and the device name is not a device.
• An unparsable value was echoed. 'garbage' → device: "garbage", against a field the contract publishes as a closed set.

The served resolver now normalises and validates, returning None for anything outside {"cpu", "cuda", "mps"}. None is chosen over raising or over a silent "cpu", because it is already a legal value for the field, it is honest, and defaulting to "cpu" "would mean a typo silently changes which device the process is believed to use, which is the _asset_max_file_bytes mistake from F-7 in a different variable." The guard is kept as a literal, not derived from the implementation, so it encodes the contract's set and cannot drift with the code:

_LEGAL_DEVICES: frozenset[str] = frozenset({"cpu", "cuda", "mps"})

Config.device_preference still returns the raw override, deliberately: it is a general-purpose property whose other callers may legitimately want the operator's literal text, and narrowing it would be a wider change than the defect warrants. The served path is the one the contract constrains, so it is the one that validates.

---

9. The tunnel agent and the Codespace launcher

9.1 deploy/codespace/serve.py — the entrypoint

The file is 25 lines and its whole job is to bind build_space_app() to $PORT:

import os

from app.space_app import build_space_app
import uvicorn

app = build_space_app()

if __name__ == "__main__":
    port = int(os.environ.get("PORT", "8000"))
    uvicorn.run(app, host="0.0.0.0", port=port)

Its docstring records the properties that make it import-safe on a CPU host with no GPU and no weights:

"build_space_app() is cheap to import: FastAPI is imported inside it and no model is loaded at module scope, so this file stays import-safe on a CPU host with no GPU and no weights present."

and it names the device resolution path: "The serving controller (via app.serving.build_serving_controller) resolves device from the SATQUERY_DEVICE env var; set it to cpu for the CPU-first adaptation."

9.2 deploy/codespace/launch.sh — what runs on every Codespace start

The script is the devcontainer's postStartCommand target. It runs three stages plus a preflight.

Stage 0 — preflight, refusing to start half-configured. The header comment states why this exists:

"A silently-broken environment is the single worst failure mode here: the server dies, nothing listens on the port, and the only external symptom is a bare 401/302 from GitHub's relay — which looks like a visibility problem."

It therefore checks the Python dependencies including httpx explicitly, and the comment records the incident:

"NOTE: httpx is checked explicitly. tunnel_agent.py imports it directly, and it was previously absent from requirements.txt — so the agent died instantly and the supervised restart loop hid the error in a log file."

if ! python -c "import yaml, pydantic, fastapi, uvicorn, httpx" 2>/dev/null; then
  echo "ERROR: Python deps are missing (need yaml, pydantic, fastapi, uvicorn, httpx)." >&2
  ...
  exit 1
fi

if ! python -c "import app.space_app" 2>/dev/null; then
  echo "ERROR: cannot import the 'app' package even with PYTHONPATH=$REPO_ROOT" >&2
  ...
  exit 1
fi

Stage 1 — the inference server, with a staleness guard. The script records a stamp of the git revision and the asset-upload environment, because _port_open alone cannot tell you what the running process was started from:

_current_stamp() {
  printf 'rev=%s asset_enabled=%s asset_dir=%s\n' \
    "$(git rev-parse HEAD 2>/dev/null || echo nogit)" \
    "${SATQUERY_ASSET_ENABLED:-}" \
    "${SATQUERY_ASSET_DIR:-}"
}

and the comment explains the failure a stale process causes:

"A stale serve process is worse than no process: it answers /v1/health and /v1/capabilities from OLD code, so the deployment looks alive while reporting the previous revision's capabilities."

The restart uses the real invocation, not the file path — pkill -f "python deploy/codespace/serve.py", because "pgrep -f serve.py would also match an editor or this script's own argv." If SIGTERM is not enough it escalates to pkill -9, and the process is launched detached:

setsid nohup python deploy/codespace/serve.py > "$SERVE_LOG" 2>&1 < /dev/null &

Stage 2 — the outbound tunnel agent, supervised. The header comment records the production incident that shaped the launch:

"setsid alone is NOT enough in Codespaces. The lifecycle shell that runs postStartCommand can still reap the process group, which showed up in production as 'the agent announced once, then vanished' — the hub then reported agent_connected=false and /api/infer fell back to the dead forwarded-port path (401 -> wake_timeout)."

The remedy is setsid + nohup + </dev/null plus a supervising wrapper that relaunches the agent if it ever exits:

setsid nohup bash -c '
  while true; do
    echo "[supervisor $(date +%H:%M:%S)] starting tunnel agent" >> "'"$TUNNEL_LOG"'"
    python deploy/codespace/tunnel_agent.py >> "'"$TUNNEL_LOG"'" 2>&1
    rc=$?
    echo "[supervisor $(date +%H:%M:%S)] tunnel agent exited rc=$rc — restarting in 5s" >> "'"$TUNNEL_LOG"'"
    sleep 5
  done
' > /dev/null 2>&1 < /dev/null &

The guard is on the process, not a port — "the agent listens on nothing" — and the supervisor itself is what gets detached, so "the agent is effectively immortal for the life of the Codespace."

Stage 3 — verify the agent actually connected. This stage exists because backgrounding with all output discarded makes a crashing agent invisible:

"Backgrounding with all output discarded means a crashing agent is completely invisible — that is exactly how a missing httpx hid itself. So we wait, then check: the process is alive, and the log shows a successful announce."

sleep 4

if ! pgrep -f "deploy/codespace/tunnel_agent.py" > /dev/null 2>&1; then
  echo "WARNING: the tunnel agent is not running. Last log lines:" >&2
  tail -n 20 "$TUNNEL_LOG" 2>/dev/null >&2 || echo "  (no log at $TUNNEL_LOG)" >&2
  ...
else
  echo "tunnel agent process is up (pid $(pgrep -f 'deploy/codespace/tunnel_agent.py' | head -1))"
  if grep -q "announced to hub" "$TUNNEL_LOG" 2>/dev/null; then
    echo "tunnel agent announced to the hub successfully"
  ...

The hub URL is a defaulted variable, so a renamed Render service can be overridden in the Codespace:

export SATQUERY_HUB_URL="${SATQUERY_HUB_URL:-https://<backend-host>}"

9.3 The asset-upload environment, and why /tmp is correct

launch.sh sets the asset variables on every start, and its comment argues the choice rather than asserting it:

"/tmp is correct here and not a compromise: the Codespace filesystem is ephemeral, handles are TTL'd (900s), and cache_max_models: 1 means an uploaded asset is consumed within one analysis, so nothing needs to outlive the process. The store creates the directory if absent."

export SATQUERY_ASSET_ENABLED="${SATQUERY_ASSET_ENABLED:-1}"
export SATQUERY_ASSET_DIR="${SATQUERY_ASSET_DIR:-/tmp/satquery-assets}"

The comment also cross-references the exact fallback path in code — "the directory is intentionally the same path the code falls back to (space_app.py:310)" — which is Path(tempfile.gettempdir()) / "satquery-assets" in app/space_app.py::_asset_root(). That is a deliberate alignment: "a deployment that set only the flag — or neither — cannot silently start writing to a barely-chosen location."

9.4 The tunnel agent file itself

Question	Answer	Status
Is deploy/codespace/tunnel_agent.py in the monorepo working tree?	No — Glob **/tunnel_agent* finds nothing	MEASURED
Is it tracked by git?	No — git ls-files deploy/ is empty; the whole deploy/ tree is untracked	MEASURED
Where does it exist?	Anish-lab-blip/SatQuery-Inference (private) — "Codespace FastAPI + deploy/codespace/tunnel_agent.py" (docs/FINAL_DELIVERY_TODO.md §1.3)	VERIFIED
What are its function names, arguments, payload shapes?	UNKNOWN — not established from the available evidence	OPEN
What is established about it?	it imports httpx; it dials SATQUERY_HUB_URL; it executes against http://127.0.0.1:8000; it logs announced to hub; it is supervised by launch.sh	VERIFIED (from launch.sh comments and greps)

This is the single largest evidence gap in this chapter, and it is recorded rather than filled in with a plausible guess. A reader who needs the agent's protocol should read SatQuery-Inference/deploy/codespace/tunnel_agent.py.

9.5 The "stale working copy" trap — B-03

B-03 is KNOWN, and it is the reason §9.4 has a gap at all:

"B-03 | Local deploy/ stale + untracked | Edits there do not deploy | all deploy tasks | edit the 3 real repos instead | KNOWN" (docs/FINAL_DELIVERY_TODO.md §5)

"Local deploy/render/main.py (532 lines, no tunnel) is superseded by SatQuery-Backend/main.py (768 lines, tunnel)." (docs/FINAL_DELIVERY_TODO.md §1.1)

"Critical: the deployed backend is not this working copy." (docs/FINAL_DELIVERY_TODO.md §1.1)

"Local deploy/ | stale/untracked | Edit the 3 real repos, not this copy." (docs/FINAL_DELIVERY_REPORT.md §6)

9.6 Repositories of record

docs/FINAL_DELIVERY_TODO.md §1.3:

Repo	Role	Deployed from
Anish-lab-blip/SatQuery-Frontend (private)	Cloudflare Pages (static)	root = local frontend/ contents
Anish-lab-blip/SatQuery-Backend (private)	Render hub + tunnel.py + codespaces.py	Render satquery-orchestrator
Anish-lab-blip/SatQuery-Inference (private)	Codespace FastAPI + deploy/codespace/tunnel_agent.py	Codespace
Anish-lab-blip/SatQuery-AI (public)	umbrella / monorepo mirror	—

9.7 Deployed revisions

Component	Repository	Branch	Revision	Host
Frontend	SatQuery-Frontend	main	2d7ae53b482d	Cloudflare Pages → satquery.pages.dev
Backend / orchestrator	SatQuery-Backend	main	89d80eaddec5	Render → <backend-host>
Inference	SatQuery-Inference	main	5a0936ace491	Codespace potential-space-trout-r4ppw969w45j2pvvw, port 8000
Public umbrella	SatQuery-AI	main	3dcabd32da41	the release home
Monorepo working copy	C:/Users/anish/satquery-ai	master	9d57aed	local only, no remote, 334 dirty entries

Source: release/repo/docs/DEPLOYMENT.md §1. This is the correct place to look up a deployed revision; the monorepo HEAD is not the deployed revision.

---

10. Deployment mechanics

10.1 Frontend → Cloudflare Pages

Staged by scripts/stage_pages.mjs, deployed with npx wrangler pages deploy. The staging run measured on 2026-09-25 (docs/DEPLOYMENT_DECISION.md §7):

files staged          : 60
total bytes           : 39,173,936 (37.36 MiB)
largest file          : assets/video/satquery-launch-50s.mp4  22,710,313 B (21.66 MiB)
25 MiB headroom left  : 3,504,087 B on the largest file
missing refs in staged : 0
external network deps : 0  (HERMETIC)
exit                  : 0

_headers and robots.txt must be force-included because no page references them; provenance.json and CREDITS.md likewise, because they are provenance records rather than assets (docs/DEPLOYMENT_DECISION.md §7).

10.2 Backend → Render

render.yaml is the blueprint (§8.2); main.py exposes app (uvicorn deploy.render.main:app --host 0.0.0.0 --port $PORT).

10.3 Inference → Codespace

deploy/codespace/serve.py serves build_space_app() on $PORT; .devcontainer/ forwards port 8000 and runs the tunnel agent on start via postStartCommand (release/repo/docs/DEPLOYMENT.md §4).

10.4 Repository writes use the GitHub Git Data API, not git push

"Repository writes are performed through the GitHub Git Data API (blob → tree → commit → PATCH ref) with sha256 byte-verification of every uploaded blob. Deletions are expressed as sha: null tree entries. This is used instead of git push so each deployed file is verified by content hash." (release/repo/docs/DEPLOYMENT.md §4)

The integrity check is recorded: "Deployed files were re-read from the GitHub API and compared byte-for-byte against the local copies: 9 files sha256 byte-identical, and the deployed HEAD re-read from the API." (release/repo/docs/DEPLOYMENT.md §4.1; docs/FINAL_DELIVERY_TODO.md §6 E-10.)

---

11. The five historical backend blockers, and how the design closes them

docs/DEPLOYMENT_DECISION.md §8 enumerated five verified backend blockers that had to be closed before any backend could boot. docs/DEPLOYMENT_TOPOLOGY.md §4 carries them forward with the active design's response.

#	Blocker (verified, old doc)	How the new topology addresses it
1	requirements.txt declared no fastapi / uvicorn / httpx / starlette	the Codespace/Render runtime installs the ASGI stack so build_space_app() and the gateway app can import
2	No code read $PORT — a platform port would be ignored	deploy/codespace/serve.py binds build_space_app() to $PORT; Render reads its own $PORT
3	Hand-rolled CORS; OPTIONS raised 405, so browser preflight failed	the gateway registers OPTIONS explicitly (or relies on Starlette's CORS middleware) so preflight succeeds
4	Module-level app = create_app() swallowed config errors into app = None	construction errors propagate (fail-fast) instead of silently leaving a dead app
5	Adapter integrity unverified on load (_adapter_sha256 computed but never compared)	the load path compares the computed digest against an expected value, or fails startup

11.1 The blockers, with their original verification

docs/DEPLOYMENT_DECISION.md §8 is the primary record, and it is more specific than the summary table:

#	Blocker	State (as recorded)
1	requirements.txt declares no fastapi / uvicorn / httpx / starlette	VERIFIED
2	No code reads $PORT — a platform-assigned port would be ignored	VERIFIED
3	CORS is hand-rolled (gateway/policy.py:429-451); policy.py:591 admits OPTIONS but routes register only GET/HEAD/POST (gateway/app.py:334-346), so Starlette raises 405 and browser preflight fails	VERIFIED
4	Module-level app = create_app() swallows config errors into app = None (gateway/app.py:683-690)	VERIFIED
5	Adapter integrity unverified on load — _adapter_sha256 is computed and stored (:258, :273) but never compared against an expected digest	VERIFIED

Blockers 3 and 4 are visible in the code this document cites. gateway/app.py's own tail is blocker 4 exactly:

try:  # pragma: no cover - depends on FastAPI being importable
    app = create_app()
except Exception:  # pragma: no cover - the sandbox path
    app = None  # type: ignore[assignment]

and deploy/render/main.py is the fail-fast counterpart — its create_app() is called at module scope with no try, so a misconfiguration raises at import:

# The ASGI object uvicorn imports: `uvicorn deploy.render.main:app`.
app = create_app()

The CORS half of blocker 3 is closed in deploy/render/main.py by registering CORSMiddleware, whose comment names the defect it fixes:

"CORS fix: CORSMiddleware answers OPTIONS preflight itself, which resolves the earlier 405 on preflight."

11.2 Status: closed by construction, not proven in production

docs/DEPLOYMENT_TOPOLOGY.md §4 is careful about the claim, and this document keeps that caution:

"They are recorded honestly here — the new infra (deploy/render/, deploy/codespace/) is in progress, so treat these as closed by construction / to be verified on first live run, not as already proven in production."

However, the live deployment has since been exercised end-to-end: docs/FINAL_DELIVERY_REPORT.md §3 records /api/health 200, /api/capabilities 200 with 6× available:true, /api/infer {} → 422 invalid_request with x-satquery-transport: tunnel, and real inference for all six tasks. So the honest composite statement is: the five blockers are closed in the deployed system as evidenced by the live behaviour recorded in the delivery documents, while docs/DEPLOYMENT_TOPOLOGY.md §4's own text still carries the earlier "in progress" framing. Where the two disagree, the dated measurement is the stronger evidence, and it is cited here rather than substituted for the source's own words.

---

12. The superseded design, and exactly what did NOT change

12.1 The historical topology

The superseded design ran inference on an HF Space with ZeroGPU (5 GPU-min/day, @spaces.GPU(duration=…) decoration) behind a Railway gateway (docs/DEPLOYMENT_TOPOLOGY.md §5; docs/DEPLOYMENT_ARCHITECTURE.md §1, §3).

Old (superseded)	New (active)
Railway (gateway/API)	Render (orchestrator / API gateway)
Hugging Face Space (inference)	GitHub Codespace (FastAPI inference)
Cloudflare Pages	Cloudflare Pages (unchanged)
Hugging Face (project/models)	Hugging Face (project card + pinned model references)

Source: docs/DEPLOYMENT_TOPOLOGY.md §1.

12.2 The three things that changed

docs/DEPLOYMENT_TOPOLOGY.md §5 enumerates them:

1. CPU-first instead of ZeroGPU. No code change was required — device_preference honours SATQUERY_DEVICE and defaults to CPU, every specialist defaults to device="cpu", and all placement is .to(device) (never .cuda()). ZeroGPU's GPU-minute quota and @spaces.GPU decoration are no longer on the critical path.
2. A real, always-buildable inference environment. A GitHub Codespace gives a reproducible container that builds and runs build_space_app() without a GPU quota or a Space's ephemeral-cold-start constraint. The wake flow (§4) replaces ZeroGPU lazy-loading as the cold-start story.
3. No GPU quota to protect at the gateway. Because there is no ZeroGPU budget, the gateway's rate/size limits remain as fairness controls, but the "never spend GPU quota on a shape-rejected request" rationale no longer dominates the design.

The CPU adaptation is independently verified in docs/DEPLOYMENT_DECISION.md §5, which lists the specific sites: core/config.py:87-91 (device_preference), specialists/vqa/model.py:234 (float16 on cuda, float32 on cpu), the per-specialist device: str = "cpu" defaults (change/specialist.py:153, change/stanet.py:641, change/vqa_specialist.py:116, grounding/remoteclip.py:111, grounding/specialist.py:682), configs/base.yaml:293 (cpu_mode_required: true), and the fact that no .cuda() call exists anywhere — all placement is .to(device).

12.3 What did NOT change

docs/DEPLOYMENT_TOPOLOGY.md §5 closes with the list, and it is the most important part of this section:

"What did NOT change: the 4-endpoint contract, the gateway responsibility table, the env-var vocabulary (only host names moved: SATQUERY_SPACE_URL → SATQUERY_UPSTREAM_URL), and the Config.hash == 78f1e3700da15aa1 freeze. The backend contract in DEPLOYMENT_ARCHITECTURE.md §1.1, §2, §3.3, §4, §5 remains authoritative."

Expanded:

Unchanged artefact	Where it lives	Why it survived the host change
The 4-endpoint contract	docs/API_CONTRACT.md; app/space_app.py; gateway/app.py::PROXIED_ROUTES	it is a client-facing contract; hosts are an implementation detail
The gateway responsibility table	docs/DEPLOYMENT_ARCHITECTURE.md §2.1	the responsibilities are the same regardless of who hosts the upstream
The env-var vocabulary	docs/DEPLOYMENT_ARCHITECTURE.md §4	only SATQUERY_SPACE_URL → SATQUERY_UPSTREAM_URL moved
The config freeze 78f1e3700da15aa1	core/config.py::Config.hash; configs/base.yaml	the deployment was changed around the config, never inside it
The gateway failure-mode table	docs/DEPLOYMENT_ARCHITECTURE.md §5	still governs, host names aside
The entrypoint requirements	docs/DEPLOYMENT_ARCHITECTURE.md §3.3	import cheaply without torch; reuse app.serving; degrade don't crash; never load a model for a metadata request; honour the config hash

12.4 The frozen paperwork

configs/deploy.yaml still describes an HF Space + Gradio + ZeroGPU target, and it is left undisturbed (docs/DEPLOYMENT_TOPOLOGY.md §3.4; docs/DEPLOYMENT_DECISION.md §4). The reasoning is structural, not sentimental, and it is a good example of why the config freeze matters:

1. Config.hash cannot move. core/config.py reads only configs/base.yaml. configs/deploy.yaml carries registry: false and is never loaded — but scripts/validate_deploy_config.py hard-fails if the deployment: block in deploy.yaml differs key-for-key from base.yaml's (assertions at :114-131). So changing zerogpu: true → false in deploy.yaml alone fails the validator, and moving base.yaml to match moves the frozen hash. Both paths are closed.
2. There is no Gradio runtime to conflict with. No import gradio, no gr.Blocks, no gr.Interface and no Gradio entrypoint exists anywhere. Gradio appears only as requirements.txt:36 and the manifest value sdk: gradio (configs/base.yaml:284). The one ZeroGPU code path — spaces.GPU(duration=duration) at app/space_app.py:165 — sits inside decorate_gpu(), which is never applied to any route; routes use plain @api.get/@api.post at :521/:549/:555/:661. The real entrypoint is FastAPI: build_space_app() at app/space_app.py:409.

"Conclusion: the frozen contract describes a Gradio Space that does not exist in code. It is frozen paperwork, not a competing deployment." (docs/DEPLOYMENT_DECISION.md §4)

configs/base.yaml still carries the frozen ZeroGPU declarations, and app/space_app.py transcribes the durations into GPU_DURATIONS:

GPU_DURATIONS: dict[str, int] = {
    "vqa": 20,
    "caption": 20,
    "grounding": 45,
    "change": 30,
    "optical_sar": 45,
    "change_vqa": 30,
}

with change_vqa reusing the change budget because adding a key of its own would move Config.hash (docs/DEPLOYMENT_ARCHITECTURE.md §3.4; app/space_app.py). And the decoration is applied conditionally, because spaces is not installed on a CPU host:

def decorate_gpu(task: str) -> Callable[[Callable[..., Any]], Callable[..., Any]]:
    ...
    spaces = _spaces_module()
    if spaces is None or not hasattr(spaces, "GPU"):
        def _identity(fn): return fn
        return _identity
    return spaces.GPU(duration=duration)

The honest status of the ZeroGPU path: "the ZeroGPU decoration has never executed here. It is specified from finding C-8 and the frozen gpu_duration_* values, and that is all it is." (app/space_app.py docstring; docs/PHASE19_FINAL_HARDENING.md.)

---

13. Failure modes and their handling

docs/DEPLOYMENT_ARCHITECTURE.md §5 is the authoritative table. Reproduced, with the active host names:

Failure	Detected by	Surface	Recovery
Upstream cold start	gateway upstream timeout	504 with recoverable: true	client retries once, manually
Model absent	capabilities[].available: false	503 model_unavailable	capability disabled in the UI
Model corrupt	ModelLoadError	503 model_load_error	defect — report it
GPU quota exhausted	allocation error	503	wait for the daily reset
Request too large	gateway size check	413	client re-encodes
Upload content type absent or refused	content-type allowlist	415	client sends a supported type; the server does not guess
Uploaded handle expired or unknown	store lookup on read	400 input_error	re-upload; handles are ephemeral by design
Asset store not configured or full	store construction / capacity check	503	⚠️ distinguishing these two needs an instrument the deployment does not expose
Asset root configured but unusable	nothing — get_asset_store() raises outside the route's try	500 text/plain on the upstream directly; the gateway masks it as 502	⚠️ bounded defect (F-12)
Framework error upstream (404/405)	nothing on the upstream	{"detail": …} upstream; the gateway masks it as an envelope	⚠️ bounded defect (F-12b) — now fixed upstream too (app/space_app.py registers the handler)
Malformed body	gateway schema validation	422	client bug
trace.inputs echoing a path	nothing	200 with a server-side path	✅ fixed (F-13) — core/controller.py::_asset_label
trace.steps[PARSE].detail["inputs"] echoing the same path	nothing	200 with a path in the PARSE step record	✅ fixed (F-14)
A construction failure's exception string reaching the client	nothing	200 with a path in result.warnings[], evidence[].payload["message"], and the registry block twice	✅ fixed (F-15) — path-scrubbed to a basename, raw detail logged server-side; four live carriers, not three
artifact_ref / result.change_map carrying a path	nothing, and only when artifact_dir is configured	200 with a path where the contract documents an artifact:// URI	✅ fixed (F-16) — refs are null, no artifact:// fabricated, explicit non-retrievable warning
change_vqa.artifact_dir configured but never read	nothing — the key is accepted and silently ignored	no surface at all	⚠️ documented, not patched (F-17)
Upstream unreachable	gateway connection error	502	report; do not silently retry analyze
Analysis exceeds budget	SpecialistTimeoutError	504, recoverable: true	offer a retry
Non-JSON response upstream	gateway parse check	502 with the upstream body logged	defect
Per-IP rate limit bypassed	not detected	no 429 is produced	fairness only; not a protection control (§2.4)

13.1 Two failure modes that the gateway and the upstream now agree on

The 404/405 and unhandled-exception rows were originally upstream holes that the gateway masked. Both are now closed on the upstream as well, so a client following the runbook to the upstream's own URL gets the same envelope as a client going through the gateway. app/space_app.py registers both handlers, and its comment records the measurement that forced it:

"Measured, direct to the Space, before this fix: GET /v1/whocares -> 404 {"detail":"Not Found"}, GET /v1/assets -> 405 {"detail":"Method Not Allowed"}, an unwrapped failure -> 500 text/plain, no envelope at all."

13.2 A saturated asset store is indistinguishable from a misconfigured one

docs/DEPLOYMENT_ARCHITECTURE.md §5.1 records finding F-11 and its resolution. POST /v1/assets answers 503 in two unrelated situations — the store is not configured, or the store is full — with the same status and the same envelope shape, so "a client and an operator cannot tell them apart from a response."

The one value that would have separated them (capacity_refusals from AssetStore.stats()) was computed on every request and read by nothing. RESOLVED 2026-09-23 by owner ruling — the unused computation was REMOVED, not given a consumer. The owner's reasoning: "a metrics surface with no reader is a cost paid on every request for an instrument nobody holds." The ambiguity itself remains, and the document says so:

"Removing the counter did NOT remove the ambiguity. The two 503 causes remain indistinguishable from a response, and the deployment still does not expose an instrument that tells them apart."

The remedy is unchanged: SATQUERY_ASSET_MAX_FILES / SATQUERY_ASSET_TTL_S if the store is saturating, and those two variables if it is unconfigured — but confirming which requires inspecting the deployment, because the response will not say.

13.3 A deployment precondition list, carried forward

docs/DEPLOYMENT_TOPOLOGY.md §6, with host names updated:

1. Cloudflare Pages project name / domain (needed for the deploy command and robots.txt sitemap).
2. Artifacts present, or capabilities shipped available: false (change head, change_vqa head, calibration JSON) — degrades honestly, not broken.
3. HF_TOKEN set on Render if the HF proxy path is used (not used in the live config).
4. Codespace .devcontainer/ forwarding :8000 and starting the tunnel agent.
5. The five blockers in §11 closed and verified on the first live run.

---

14. What is deliberately absent from the deployment

docs/DEPLOYMENT_ARCHITECTURE.md §6 records the exclusions so that omission is not mistaken for oversight. From plan §73/§74:

• No Kubernetes, no Docker swarm. Render plus one Codespace is the whole fleet.
• No Kafka, no Redis cluster, no queue. Requests are synchronous.
• No autoscaling. The free tier has a fixed quota; autoscaling cannot raise it.
• No multi-tenant isolation, no auth, no user accounts.
• No second VLM and no foundation-model retraining.
• No vector database. The retriever-free RAG decision is separate and upstream.
• No database, no session store at the gateway (docs/DEPLOYMENT_ARCHITECTURE.md §2.2).

docs/FRONTEND_INTEGRATION.md §7 adds the client-side counterpart: no login screen, because "There is none to build (plan §74)."

---

15. What is NOT RUN / OPEN / BLOCKED for this topic

Item	Status	Note
B-07 transient tunnel-agent gaps	OPEN	patch prepared, not deployed; worst case ≈ 249 s (§6)
B-02 codespace_name trailing \n	OPEN (cosmetic)	reporting only; the wake path strips (§7.4)
B-03 local deploy/ stale + untracked	KNOWN	the working copy is not the deployed source (§9.5)
B-06 Render free-tier sleep / Codespace idle 30 min	KNOWN	cold start delay; documented, not hidden (§5)
Tunnel agent source (tunnel_agent.py)	UNKNOWN	not in the monorepo; UNKNOWN — not established from the available evidence (§9.4)
Cold-start latency distribution	NOT MEASURED	"tens of seconds" is a documented expectation; no distribution exists (§5)
Throughput / concurrency characterisation	NOT RUN	docs/FRONTEND_INTEGRATION.md §9: "Latency is not characterized."
ZeroGPU decoration execution	NOT RUN	never executed anywhere; CPU path only (app/space_app.py; docs/PHASE19_FINAL_HARDENING.md)
Sequential-request test under cache_max_models=1	NOT DONE — environment-blocked	requires a reachable upstream (docs/DEPLOYMENT_ARCHITECTURE.md §7)
Gateway's rate limiter as an abuse control	REJECTED	ruled fairness-only, 2026-09-23 (§2.4)
HF_TOKEN proxy path	not used live	absent from the live Render config (§8.1)
SATQUERY_UPSTREAM_URL	not used live	absent from the live Render config (§8.1)
A second copy of the capability table at the gateway	REJECTED	docs/DEPLOYMENT_ARCHITECTURE.md §2.2
End-to-end benchmark of the deployed stack	does not exist	no system-level accuracy is claimed anywhere
Asset-store 503 disambiguation instrument	absent	removed by ruling; ambiguity remains (§13.2)

---

16. Where the evidence lives

Claim	Source
four tiers, host names, tunnel direction	docs/DEPLOYMENT_TOPOLOGY.md §1, §2; docs/FINAL_DELIVERY_TODO.md §1.2
gateway rationale (three reasons)	docs/DEPLOYMENT_ARCHITECTURE.md §1.1
gateway responsibility table	docs/DEPLOYMENT_ARCHITECTURE.md §2.1
what the gateway must NOT do	docs/DEPLOYMENT_ARCHITECTURE.md §2.2
CORS assembly + wildcard refusal	deploy/render/main.py::_allowed_origins, _DEV_ORIGINS, _PRODUCTION_ORIGINS
CORS header filtering assertion	gateway/app.py::_proxy (F-2)
per-file cap 4,194,304 B	gateway/policy.py:221; app/space_app.py::_asset_max_file_bytes
body cap 8 MiB + F-6 measurement	docs/DEPLOYMENT_ARCHITECTURE.md §4
rate-limiter ruling + measurement	docs/DEPLOYMENT_ARCHITECTURE.md §5.2
no-retry rule	gateway/app.py::_proxy; docs/DEPLOYMENT_TOPOLOGY.md §2
request-id injection	gateway/app.py::_proxy
404/405 envelope handler	gateway/app.py; app/space_app.py
error-envelope shape	docs/DEPLOYMENT_ARCHITECTURE.md §2.3
orchestrator error classes + statuses	deploy/render/main.py
transport-failure classification	gateway/app.py::_transport_failure_detail (F-15c)
proxied / costly / blocked route tuples	gateway/app.py::PROXIED_ROUTES, COSTLY_ROUTES, BLOCKED_ROUTES
forwarded port returns 302	docs/DEPLOYMENT_TOPOLOGY.md measured note; release/repo/docs/DEPLOYMENT.md §7
tunnel rationale for private repos	deploy/codespace/launch.sh header comment
x-satquery-transport as proof	tests/unit/test_frontend_live_wiring.py; docs/FINAL_DELIVERY_TODO.md §6 E-03
wake flow	docs/DEPLOYMENT_TOPOLOGY.md §2; deploy/render/main.py::ensure_codespace_up
X-SatQuery-State header	deploy/render/main.py::infer
cold start, documented not hidden	docs/DEPLOYMENT_TOPOLOGY.md §2; docs/FRONTEND_INTEGRATION.md §6
Codespace idle 30 min	docs/FINAL_DELIVERY_TODO.md §6 E-04
B-07 root shape + ≈249 s	docs/FINAL_DELIVERY_TODO.md §5; DELIVERY_REPORT_2026-09-25.md §4
B-07 patch contents + verification	DELIVERY_REPORT_2026-09-25.md §4; docs/FINAL_DELIVERY_TODO.md §6 E-12
live health payload	release/repo/docs/DEPLOYMENT.md §2
completed readings 97 / 314 / 338	release/repo/docs/DEPLOYMENT.md §2; docs/FINAL_DELIVERY_TODO.md §1.4; docs/FINAL_DELIVERY_REPORT.md §3
B-02 cosmetic	docs/FINAL_DELIVERY_REPORT.md §6; docs/FINAL_DELIVERY_TODO.md §5
live Render env vars	docs/DEPLOYMENT_TOPOLOGY.md header note; release/repo/docs/DEPLOYMENT.md §3.1
blueprint env vars	render.yaml
Codespace env vars	.devcontainer/devcontainer.json; deploy/codespace/launch.sh
env-var vocabulary + F-7/F-8/F-9	docs/DEPLOYMENT_ARCHITECTURE.md §4
serve.py entrypoint	deploy/codespace/serve.py
launcher stages 0–3	deploy/codespace/launch.sh
tunnel agent existence + gap	docs/FINAL_DELIVERY_TODO.md §1.3; Glob/git ls-files on the monorepo
repos of record	docs/FINAL_DELIVERY_TODO.md §1.3
deployed revisions	release/repo/docs/DEPLOYMENT.md §1
staging measurement	docs/DEPLOYMENT_DECISION.md §7
Git Data API + sha256 verification	release/repo/docs/DEPLOYMENT.md §4, §4.1
five blockers + original verification	docs/DEPLOYMENT_DECISION.md §8; docs/DEPLOYMENT_TOPOLOGY.md §4
superseded design + what did not change	docs/DEPLOYMENT_TOPOLOGY.md §5
frozen paperwork	docs/DEPLOYMENT_DECISION.md §4; docs/DEPLOYMENT_TOPOLOGY.md §3.4
GPU_DURATIONS	app/space_app.py
failure modes	docs/DEPLOYMENT_ARCHITECTURE.md §5
asset-store ambiguity (F-11)	docs/DEPLOYMENT_ARCHITECTURE.md §5.1
deliberate exclusions	docs/DEPLOYMENT_ARCHITECTURE.md §6; docs/FRONTEND_INTEGRATION.md §7

---

Continue to 03 — Request lifecycle.