release: add docs/ARCHITECTURE.md

docs/ARCHITECTURE.md

@@ -1,201 +1,145 @@
-# Architecture
+# SatQuery AI — Architecture
 
-**Status vocabulary used throughout these docs:** `IMPLEMENTED` (code exists and runs) ·
-`VERIFIED` (checked against evidence) · `MEASURED` (a number was produced) · `ATTEMPTED`
-(tried, outcome recorded) · `NOT RUN` · `BLOCKED` · `DEFERRED` · `REJECTED` (tried and
-explicitly not accepted). Every architectural claim below carries one of these tags.
+This is the architecture reference for SatQuery AI. It is written as a **hub plus ten deep
+sub-documents**, because the system is large enough that a single file would either be superficial or
+unreadable.
 
----
-
-## 1. What the system is
+> **Status vocabulary used everywhere in these docs:** `IMPLEMENTED` (the code exists and runs) ·
+> `VERIFIED` (checked against evidence) · `MEASURED` (a number was produced) · `ATTEMPTED` (tried,
+> outcome recorded) · `NOT RUN` · `BLOCKED` · `DEFERRED` · `REJECTED` (tried and explicitly not
+> accepted) · `OPEN` (known, unresolved) · `RESOLVED` · `CLOSED`.
+>
+> Every substantive claim in this set carries one of these tags, and every non-obvious claim cites
+> the file it came from.
 
-SatQuery AI answers natural-language questions about satellite imagery. It is a **router +
-specialists** system: a small intent router reads the question, dispatches it to one of six
-task specialists, and assembles the specialist's output into a single structured
-`ResultEnvelope` that the browser renders as an answer, an evidence list, and a confidence.
+---
 
-It is not a single end-to-end vision-language model. The only learned components are:
+## 1. The thesis in one paragraph
 
-- a **50,822-parameter router adapter** over a frozen `all-MiniLM-L6-v2` sentence encoder;
-- four trained **task heads/adapters** (change, change_vqa, optical-SAR fusion, grounding);
-- one **LoRA adapter** over a frozen `SmolVLM-500M-Instruct` for caption/VQA.
+SatQuery AI answers natural-language questions about satellite imagery. It is **not** one large
+vision-language model. It is a **router plus specialists** system: a small learned router reads the
+question and decides *which capability* is being asked for; the controller then runs exactly one
+specialist; an evidence engine aggregates what that specialist produced; a confidence stage attaches
+a calibrated (or honestly uncalibrated) number; and the whole thing is returned as one typed
+`ResultEnvelope`. The only trained parameters in the system are six small modules sitting on frozen,
+publicly-pinned backbones.
 
-Everything else is frozen, publicly-pinned backbone weights resolved at run time.
+`docs/ARCHITECTURE_FREEZE.md` section 5 gives each layer exactly one verb, and the whole design
+follows from that sentence:
 
-## 2. Deployment topology (IMPLEMENTED, VERIFIED)
+> router **understands**; policy engine **decides**; specialists **compute**; VLM **explains**;
+> evidence engine **proves**.
 
-The shipped system is a four-tier chain. **Status: VERIFIED live on 2026-09-25** — the health,
-capabilities and infer endpoints were probed against the running deployment.
+## 2. Sub-documents
 
-```
-Browser
-  │ HTTPS
-  ▼
-Cloudflare Pages  —  satquery.pages.dev          (static frontend, 11 pages)
-  │ HTTPS/JSON
-  ▼
-Render            —  satquery-backend-m4yv.onrender.com   (orchestrator / gateway)
-  │ outbound long-poll tunnel  POST /tunnel/agent
-  ▼
-GitHub Codespace  —  FastAPI inference, port 8000 (CPU)
-  │ build_space_app()
-  ▼
-Specialists: SmolVLM · RemoteCLIP · MiniLM · CROMA · STANet
-```
+| # | Document | What it covers |
+|---|---|---|
+| 01 | [System overview](architecture/01-system-overview.md) | the thesis, the component inventory, the frozen-backbone strategy, what is deliberately absent |
+| 02 | [Deployment topology](architecture/02-deployment-topology.md) | the four tiers, the gateway, the outbound tunnel, wake flow, cold start, `transport_mode` |
+| 03 | [Request lifecycle](architecture/03-request-lifecycle.md) | the nine-state controller, validation rules, modality inference, tiling |
+| 04 | [Router](architecture/04-router.md) | frozen MiniLM, the five-head adapter, `interpret()` vs `chooseTask()`, the lexical fallback, the label space |
+| 05 | [Specialists](architecture/05-specialists.md) | all six tasks: entry points, preprocessing, postprocessing, outputs |
+| 06 | [Evidence and confidence](architecture/06-evidence-and-confidence.md) | the evidence schema, the aggregation pipeline, temperature scaling, the eight execution events |
+| 07 | [Configuration freeze](architecture/07-configuration-freeze.md) | the registry, the enforced invariants, the config hash, why it is frozen |
+| 08 | [API contract](architecture/08-api-contract.md) | the four endpoints, the envelopes, error codes, transport headers |
+| 09 | [Frontend](architecture/09-frontend.md) | the static pages, the Analyze console, real-vs-preview, platform traps |
+| 10 | [Observability and operations](architecture/10-observability-and-ops.md) | health, counters, traces, what is and is not observed |
+
+## 3. The system at a glance
 
 ```mermaid
-flowchart LR
-  U[Browser] -->|HTTPS| CF["Cloudflare Pages<br/>static frontend"]
-  CF -->|"HTTPS JSON<br/>/api/health · /api/capabilities · /api/infer"| R["Render<br/>orchestrator / gateway"]
-  R -->|"outbound long-poll<br/>POST /tunnel/agent"| C["GitHub Codespace<br/>FastAPI inference :8000"]
-  C --> S[(SmolVLM · RemoteCLIP<br/>MiniLM · CROMA · STANet)]
-  C -->|ResultEnvelope| R
-  R -->|"envelope + error translation"| CF
+flowchart TB
+  subgraph Client
+    U[Browser]
+  end
+  subgraph Static["Static tier"]
+    CF["Cloudflare Pages<br/>satquery.pages.dev"]
+  end
+  subgraph Gateway["Gateway tier (Render)"]
+    R["satquery-orchestrator<br/>validate · CORS · limits · timeouts · errors"]
+  end
+  subgraph Inference["Inference tier (GitHub Codespace, CPU)"]
+    A["FastAPI · build_space_app()"]
+    CTRL["Controller (9-state FSM)"]
+    ROUTER["Router<br/>MiniLM + 5-head adapter"]
+    SPEC["Specialists<br/>vqa · caption · grounding · change · change_vqa · optical_sar"]
+    EV["Evidence engine<br/>dedup · order · renumber · cap"]
+    CONF["Confidence<br/>temperature scaling"]
+    A --> CTRL --> ROUTER --> SPEC --> EV --> CONF
+  end
+  U -->|HTTPS| CF
+  CF -->|"HTTPS JSON /api/*"| R
+  R -->|"outbound long-poll POST /tunnel/agent"| A
+  CONF -->|ResultEnvelope| R
+  R -->|envelope + error translation| CF
 ```
 
-### 2.1 Why a gateway and a tunnel
-
-- **The gateway (Render)** is deliberately thin and stateless: schema validation, size limits,
-  per-IP rate limiting, a CORS allowlist (**never** `*`), request IDs, timeouts, secret custody,
-  and translation of upstream failures into the documented error envelope. It is **not** a model
-  host and has no database, auth, or queue.
-- **The tunnel** exists because the inference host (a Codespace) is not publicly addressable for
-  this private repository — a forwarded port returns `302`. Instead the Codespace runs
-  `tunnel_agent.py`, which **dials out** to `POST /tunnel/agent` and long-polls. Work is executed
-  against `http://127.0.0.1:8000` locally. This inverts the usual direction: the inference host
-  needs no inbound firewall hole.
-
-> **Status:** IMPLEMENTED and VERIFIED. `GET /api/health` reported `tunnel.agent_connected:true`
-> with a non-zero `completed` counter on 2026-09-25.
-
-### 2.2 Cold start and the `transport_mode: auto` fallthrough (KNOWN ISSUE, OPEN)
-
-The Render free tier sleeps when idle and the Codespace may be stopped. Cold start is therefore
-tens of seconds and is **documented, not hidden**. A timeout on the tunnel path is surfaced to the
-client as a recoverable error.
+## 4. Cross-cutting principles
 
-`SATQUERY_TRANSPORT=auto` means: try the tunnel; if it does not answer within
-`SATQUERY_TUNNEL_TIMEOUT_S` (150 s), **fall through to the forward path**. The forward path to a
-private repo returns `302` quickly, but the wake step still consumes `SATQUERY_WAKE_TIMEOUT_S`
-(120 s) first — so a worst-case failed request can take ≈ 249 s. This is the root of the observed
-"transient tunnel gap" and is recorded as **OPEN** (not fixed) in
-[`LIMITATIONS.md`](LIMITATIONS.md). A deployed fix (`codespace_name` newline handling on the
-wake path) was authored and pushed separately; the fallthrough itself is by design.
+These recur in every sub-document and are the reason the code looks the way it does.
 
-## 3. Request lifecycle (IMPLEMENTED, VERIFIED)
+### 4.1 One config system, no magic numbers
 
-The controller is a nine-state machine, declared in `configs/base.yaml`:
+Every tunable value lives in `configs/base.yaml`. The loader (`core/config.py`) validates it against
+the frozen architecture and hashes it. **No number is hard-coded in Python.** This is enforced
+socially and structurally: a reviewer who finds a literal in a specialist has found a bug.
 
-```
-RECEIVE → PARSE → VALIDATE → PLAN → PREPROCESS → EXECUTE → AGGREGATE → VERIFY → RESPOND
-```
-
-1. **RECEIVE / PARSE** — the query and 1–2 image assets are decoded.
-2. **VALIDATE** — modality inference from band count, dimension equality for pairs, byte cap
-   (`4,194,304` bytes/file), GeoTIFF acceptance for the optical-SAR path.
-3. **PLAN** — the router embeds the query with the frozen MiniLM encoder and the trained adapter
-   predicts a task + modality. Dispatch is asset-count-aware: a single image cannot be a change
-   task.
-4. **PREPROCESS** — percentile normalisation for optical (2/98), dB clip for SAR (−30…+5 dB),
-   tiling for large scenes.
-5. **EXECUTE** — the chosen specialist runs.
-6. **AGGREGATE** — the specialist's raw output becomes `Evidence` items
-   (`Region` / `ChangeRegion`).
-7. **VERIFY** — confidence is computed; temperature scaling is applied from
-   `calibration_v001.json`.
-8. **RESPOND** — a `ResultEnvelope` is assembled and returned.
-
-### 3.1 The eight execution events
-
-The frontend renders a live trace from eight named events (source of truth:
-`frontend/assets/js/core.js`, `SQ.EVENT_NAMES`):
-
-| # | Event | Emitted when |
-|---|---|---|
-| 1 | `QUERY_RECEIVED` | the request is accepted |
-| 2 | `QUERY_UNDERSTOOD` | the router produces task + modality |
-| 3 | `ROUTE_SELECTED` | the specialist is chosen |
-| 4 | `SPECIALIST_STARTED` | specialist execution begins |
-| 5 | `SPECIALIST_COMPLETED` | specialist returns |
-| 6 | `EVIDENCE_GENERATED` | evidence items are built |
-| 7 | `CONFIDENCE_COMPUTED` | confidence (and calibration) is computed |
-| 8 | `RESULT_ASSEMBLED` | the envelope is finalised |
-
-**Live runs emit all eight; the offline preview path emits none of the specialist events and is
-labelled as a preview in the UI.** Measured trace fill on every live run: **94.4444 %**.
+### 4.2 Frozen backbones, trained modules
 
-## 4. Two reading paths: `interpret()` vs `chooseTask()` (IMPLEMENTED, VERIFIED)
+No backbone is fine-tuned. `all-MiniLM-L6-v2`, `SmolVLM-500M-Instruct`, `RemoteCLIP ViT-B/32` and
+`CROMA-base` are all pinned **by revision** and fetched from the Hub at run time. What this project
+trains is small: a 50,822-parameter router adapter, four heads, and one LoRA adapter. This is what
+makes the system CPU-runnable.
 
-There are two separate functions, and they see different information:
+### 4.3 Typed contracts between every layer
 
-- **`interpret()`** — produces the *human-readable* interpretation shown in the console. It is
-  **asset-count-blind**: it only sees the query text.
-- **`chooseTask()`** — performs the *dispatch*. It is **asset-count-aware**: it knows how many
-  assets are attached and will not route a single-image query to a two-image task.
+`core/schemas.py` is the binding contract. No specialist may invent its own result shape; every
+specialist returns a `SpecialistResult`. The schemas carry validators that encode real findings —
+for example `Evidence` **refuses** spatial coordinates without a `coordinate_system`, because a bare
+box is meaningless.
 
-This asymmetry is real and explains a documented behaviour: for the query *"What changed between
-the earlier and later image?"* with a single asset attached, the console can *read* `change` while
-dispatch correctly falls back to `change_vqa`. This is intentional, not a bug, and is covered in
-[`RESEARCH_NOTES.md`](RESEARCH_NOTES.md).
+### 4.4 Honest degradation over confident fabrication
 
-## 5. The four-endpoint inference contract (IMPLEMENTED, VERIFIED)
+The system is built so that "we could not do this" is representable and preferred to a plausible
+guess. Concretely:
 
-The Codespace exposes exactly four routes; the gateway mirrors each under `/api/*`:
+- a missing calibration artifact yields `method="uncalibrated"`, `calibrated=None` — **never** a
+  fabricated fitted number;
+- a missing optional artifact **degrades** a capability rather than crashing the service;
+- a *corrupt* artifact **raises**, because silently treating a corrupt file as "no file" would hide
+  an operational defect;
+- the evidence engine records `dropped_over_limit` rather than silently truncating;
+- the grounding specialist marks a result `degraded` when it produces no localisation.
 
-| Codespace | Gateway | Cost | Purpose |
-|---|---|---|---|
-| `GET /v1/health` | `GET /api/health` | cheap | liveness, tunnel state, counters |
-| `GET /v1/capabilities` | `GET /api/capabilities` | cheap | per-task availability |
-| `POST /v1/analyze` | `POST /api/infer` | **costly** | run a query |
-| `POST /v1/assets` | `POST /api/assets` | **costly** | stage an asset handle |
+### 4.5 Reproducibility is a structural property
 
-The gateway must **not** retry `POST /api/infer` on its own — a retry would consume inference a
-second time. The client decides on retry. Errors are translated into a stable envelope with a
-machine code (`invalid_request`, `tunnel_offline`, …) and a `recoverable` flag.
+`EvidenceEngine.aggregate` is pure and deterministic — no clock, no RNG, no I/O. Evidence ids are
+assigned from *sorted position*, not input order, so the same inputs produce byte-identical output.
+`evidence_digest()` exists so that reproducibility is a test assertion rather than a hope.
 
-**Verified:** `POST /api/infer {}` returns `422 invalid_request` with header
-`x-satquery-transport: tunnel`.
+### 4.6 No chain-of-thought anywhere
 
-## 6. The frozen configuration (VERIFIED)
+`ExecutionTrace` records **observable facts only** — states, timings, counts, config hash, model
+refs. There is no field for model reasoning and no LLM-generated confidence. This is a deliberate
+constraint from the architecture freeze, not an omission.
 
-All tunables live in `configs/base.yaml`; **no magic numbers in Python**. The loader validates
-invariants and computes a hash. The frozen hash is:
+## 5. What the system deliberately does not have
 
-```
-78f1e3700da15aa1
-```
-
-The loader **refuses to run** a config that violates a recorded invariant — for example
-`fusion.input_dim == 3 * croma.encoder_dim + croma.optical_channels + croma.sar_channels`
-(2318) and `grounding_head.feature_dim == 4 * grounding.encoder_projected_dim` (2048). A mismatch
-in the latter is a *silent* shape error otherwise (torch raises only later, after features are
-cached), so it is enforced at load time.
+| Absent | Why |
+|---|---|
+| Database, auth, queue | the gateway is stateless by design |
+| GPU requirement | device is chosen via `SATQUERY_DEVICE`; all placement is `.to(device)` |
+| Gradio GUI | the frontend is a separate static tier; `app/space_app.py` serves JSON only |
+| End-to-end benchmark | none exists; none is claimed |
+| Chain-of-thought | traces carry observable facts only |
+| Backbone redistribution | backbones are fetched, pinned by revision |
 
-Editing `configs/base.yaml` moves the hash, which invalidates every artifact keyed to it. This is
-why the stale `configs/deploy.yaml` (which still describes an HF-Space/ZeroGPU target) is left
-**undisturbed as frozen paperwork** — it is never read at run time.
+## 6. Where to start reading
 
-## 7. Component map
-
-| Layer | Path (in the source repo) | Notes |
-|---|---|---|
-| Frontend | `frontend/` | static, staged by `scripts/stage_pages.mjs` |
-| Gateway | `deploy/render/` | `main.py` exposes `app`; `render.yaml` blueprint |
-| Inference entry | `deploy/codespace/serve.py` | serves `build_space_app()` on `$PORT` |
-| Inference app | `app/space_app.py` | `build_space_app()`; 4-endpoint contract |
-| Composition root | `app/serving.py` | reuses the local serving path |
-| Specialists | `specialists/` | vqa/caption (SmolVLM), grounding (RemoteCLIP), change (STANet), optical_sar (CROMA) |
-| Router | `router/` | frozen MiniLM + trained adapter |
-| Config | `configs/base.yaml` | authoritative registry; hashed |
-| Artifacts | `artifacts/` | weights, metrics, calibration, reports |
-
-> **Caveat.** `deploy/` inside the monorepo is **stale and untracked**; the deployed sources live in
-> three separate private repositories. See [`DEPLOYMENT.md`](DEPLOYMENT.md).
-
-## 8. What is deliberately absent
-
-- **No database, no auth, no queue.** The gateway is stateless by design.
-- **No GPU requirement.** Device is selected via `SATQUERY_DEVICE`; all placement is `.to(device)`,
-  never `.cuda()`. The shipped deployment runs CPU-only.
-- **No system-level end-to-end benchmark.** There is no measured end-to-end accuracy number, and
-  none is claimed. Per-specialist metrics exist and are reported in [`BENCHMARKS.md`](BENCHMARKS.md).
+- **New to the project** → [01 System overview](architecture/01-system-overview.md), then
+  [03 Request lifecycle](architecture/03-request-lifecycle.md).
+- **Running it** → [`DEPLOYMENT.md`](DEPLOYMENT.md) and
+  [02 Deployment topology](architecture/02-deployment-topology.md).
+- **Auditing the numbers** → [`BENCHMARKS.md`](BENCHMARKS.md) and [`EVALUATION.md`](EVALUATION.md).
+- **Understanding the router defect** → [04 Router](architecture/04-router.md) and
+  [`RESEARCH_NOTES.md`](RESEARCH_NOTES.md).