release: add docs/RESEARCH_NOTES.md

docs/RESEARCH_NOTES.md

@@ -0,0 +1,204 @@
+# Research Notes
+
+Engineering findings, negative results and design decisions that would otherwise be lost. Each was
+learned by **probe or execution**, not by assumption, and each is recorded so it is not rediscovered.
+
+**Status tags:** `MEASURED` · `RESOLVED` · `REJECTED` · `OPEN` · `ATTEMPTED`.
+
+---
+
+## 1. Findings that changed the code
+
+### F4-1 — the MiniLM tokenizer ceiling (`MEASURED`)
+
+The MiniLM tokenizer's own ceiling is **256** (verified by probe). The project truncates to **128** —
+a deliberate truncation *well inside* the ceiling, not the model limit. Satellite queries are short;
+halving the sequence halves attention cost for no measurable accuracy loss. The encoder **asserts**
+`max_length ≤ 256`, because truncating above the ceiling is a silent no-op.
+
+### F4-2 — the router needs no GPU (`MEASURED`)
+
+The encoder is frozen, so embeddings are **cached** and the 50,822-parameter adapter trains on cached
+vectors. **Measured on CPU: 20 epochs / 4,096 vectors in 0.28 s.** No GPU required.
+
+### F4-3 — splits must be by group (`MEASURED`)
+
+Splits are by **group** (template / hard-negative family), never by example. Hard-negative families
+are placed in the **test** split so their accuracy measures generalisation rather than memorisation.
+Splitting by example would leak template variants across the boundary.
+
+### F5-1 — `AutoModelForVision2Seq` does not exist (`MEASURED`)
+
+In transformers 5.17.0, `AutoModelForVision2Seq` **does not exist** (not merely deprecated);
+`AutoModelForImageTextToText` is present. The loader is resolved by **feature detection**, never
+hardcoded.
+
+### F5-2 — the processor cost overrun is ~17×, not 4× (`MEASURED`)
+
+The processor's default `longest_edge` is **2048**, which upscales 512-px tiles **4×** and then splits
+them (`do_image_splitting=True`) into **17 sub-images**:
+
+| Setting | `pixel_values` | prompt tokens |
+|---|---|---|
+| default | `(1, 17, 3, 512, 512)` | 1142 |
+| pinned (`processor_longest_edge: 512`) | `(1, 1, 3, 512, 512)` | — |
+
+The plan estimated a 4× cost overrun; the real figure is **~17×**. The value **must** be set
+explicitly on the processor at construction time.
+
+### F5-3 — prompts must go through the chat template (`MEASURED`)
+
+SmolVLM requires one `<image>` token per image in the prompt. Hand-written prompt strings raise
+`ValueError`. Prompts are always built through `processor.apply_chat_template()`.
+
+### P7-1 — the RemoteCLIP projected dimension is 512 (`MEASURED`)
+
+The RemoteCLIP ViT-B/32 transformer width is **768**, but `visual.proj` maps to a **projected** dim of
+**512**. The grounding head's per-cell feature is `4 × 512 = 2048`, declared in config so
+`core/config.py` can validate the head **without importing torch**.
+
+### C-1 — the availability mask is consumed by the head, not by CROMA (`MEASURED`)
+
+CROMA always sees the canonical channel counts (12 optical, 2 SAR). The availability mask is applied
+by the **fusion head** (`input_dim = 3 × 768 + 12 + 2 = 2318`), not by the encoder.
+
+### C-6 — T4 is SM 7.5, so training uses fp16, not bf16 (`MEASURED`)
+
+The training precision is **fp16** because the target GPU (T4) is compute capability 7.5. bf16 is
+not available there.
+
+### C-7 — `image_resolution % 8 == 0` (`MEASURED`)
+
+CROMA requires `image_resolution % 8 == 0`. The native value **120** yields 225 patches.
+
+### C-8 — ZeroGPU does not support `torch.compile` (`MEASURED`)
+
+`torch.compile` must never be enabled on the (historical) ZeroGPU target.
+
+### C-9 — STANet change hyperparameters are upstream-verified (`MEASURED`)
+
+Change detection uses STANet-style architecture with upstream-verified hyperparameters (PAM
+self-attention, ResNet-18 encoder).
+
+---
+
+## 2. The grounding resolution decision — a pre-registered rejection (`REJECTED`)
+
+**Question:** should grounding decode at 448 or 224?
+
+**Answer: 224. 448 was rejected** — and the rejection is notable because it was *pre-registered* and
+then *confirmed* by a paired test:
+
+| Comparison (over identical samples, n = 16,159) | 448 vs 224 |
+|---|---|
+| mean best IoU | **−0.0147** |
+| recall@0.5 | −0.0022 |
+| recall@0.10 | −0.0699 |
+| recall@0.25 | −0.0243 |
+| latency | **1.59×** |
+| paired 95 % CI | [−0.0160, −0.0134] |
+| paired t | **−22.63** |
+| 448 better on | 8.5 % of records |
+| 448 worse on | **20.9 %** of records |
+
+448 lost on **every** axis. The pre-registered decision rule and the paired test **agree** on 224.
+This is a model of how a resolution decision should be made: declared in advance, then tested.
+
+---
+
+## 3. The router defect — a real bug, found and fixed
+
+### 3.1 Symptom
+
+The query *"Where are the built-up areas in this image?"* collapsed to **`vqa`** and answered
+**"River"** — instead of routing to `grounding`. A second query, *"Where is the new airport?"*,
+behaved the same way.
+
+### 3.2 Root cause
+
+Two functions with different information:
+
+- **`interpret()`** — produces the console's *reading*; **asset-count-blind** (text only).
+- **`chooseTask()`** — performs *dispatch*; **asset-count-aware**.
+
+The defect was in the dispatch path's handling of spatial/lexical cues, so region queries fell
+through to the generic VQA specialist. See [`ARCHITECTURE.md`](ARCHITECTURE.md) §4.
+
+### 3.3 Fix and verification (`RESOLVED`)
+
+The fix was deployed to `SatQuery-Frontend` and validated by **three independent live passes**:
+
+| Pass | Deployed HEAD | Result |
+|---|---|---|
+| 1 | `ff46eba42b18` + `d413d3672311` | 8/8 |
+| 2 | `2d7ae53b482d` | 8/8 |
+| 3 | `2d7ae53b482d` | 8/8 |
+
+Both defect queries now dispatch to `grounding`:
+
+| Query | Run id | Dispatched |
+|---|---|---|
+| Where are the built-up areas in this image? | `run_467ffa406f22` | `grounding` |
+| Where is the new airport? | `run_46980ba55c62` | `grounding` |
+
+24 live runs, 24 correct dispatches, **0 mock nodes**. Screenshots are in
+[`../screenshots/`](../screenshots/).
+
+---
+
+## 4. The harness false-positive — caught before it could lie
+
+An earlier live-validation harness typed queries with **synthetic CDP key events**, which Chrome
+**silently drops when the window lacks OS focus**. The harness therefore dispatched the page's
+*default* query and still recorded a "result" — a **false pass**.
+
+**Fix:** the current harness **asserts form state before dispatch** (`q_ok`, `obs_ok`, `t0_ok`), and
+uses deterministic query entry (`js()` value-set + `type_text()` via CDP `Input.insertText`).
+
+**Independent check:** the earlier 8/8 run was re-examined and confirmed **not** infected — its
+answers were query-specific and the query text was embedded in the answers. The failure mode is
+recorded because it is exactly the silent false-positive an evaluation harness must never have.
+
+---
+
+## 5. The `transport_mode: auto` fallthrough (`OPEN`)
+
+`SATQUERY_TRANSPORT=auto` tries the tunnel, then falls through to the forward path on timeout. The
+forward path to a **private** repo returns `302` quickly, but the wake step still consumes
+`SATQUERY_WAKE_TIMEOUT_S` (120 s) first — so a worst-case failed request takes ≈ 249 s
+(150 + 120). This is the root shape of the observed transient tunnel gap. Recorded as **OPEN**; a
+deployed fix for the `codespace_name` newline on the wake path was authored separately.
+
+---
+
+## 6. The `interpret()` / `chooseTask()` asymmetry — intentional (`RESOLVED`)
+
+For *"What changed between the earlier and later image?"* with **one** asset attached, the console
+**reads** `change` while dispatch correctly falls back to **`change_vqa`**. This is not a bug: the
+reading describes the question's intent, the dispatch respects what can actually be computed with the
+assets present. It is documented so it is not mistaken for a defect.
+
+---
+
+## 7. Environment findings (would otherwise cost hours)
+
+| Finding | Detail |
+|---|---|
+| **Dead proxy in the sandbox** | outbound calls need `--noproxy '*'` (curl) or `ProxyHandler({})` (Python). |
+| **pytest is only in the repo venv** | `.venv/Scripts/python.exe`; a bare `pytest` misses it. |
+| **Full-suite pytest trips a bulk-delete guard** | sandbox-specific; affects `test_safe_delete_shim`. |
+| **Cloudflare `_headers` concatenate** | two matching rules are merged, not overridden; Chromium takes the **first** `max-age`. |
+| **Cloudflare 308-redirects `X.html` → `/X`** | reference the extensionless path. |
+| **A forwarded Codespace port returns `302`** for a private repo | this is *why* the tunnel exists. |
+| **Chrome drops synthetic CDP key events without OS focus** | the harness false-positive (§4). |
+| **`browser-use` block-buffers stdout** even when redirected | needs explicit line buffering to stream. |
+
+---
+
+## 8. The BigEarthNet format contradiction (`ATTEMPTED`, reported not resolved)
+
+The BigEarthNet data format **contradicts the original plan**. This was **reported rather than
+silently patched**, because quietly changing the preprocessing would move the frozen config hash. The
+BigEarthNet documentation does not specify a percentile stretch; this project nevertheless applies
+percentile normalisation (2/98) to match the CROMA contract. That is a **deliberate, documented
+choice**, not an upstream fact. See [`DATASETS.md`](DATASETS.md) §5.3.