docs/MODELS.md

# Models — deep reference

**Status tags used on every substantive claim:** `IMPLEMENTED` · `VERIFIED` · `MEASURED` · `ATTEMPTED`
· `NOT RUN` · `BLOCKED` · `DEFERRED` · `REJECTED` · `OPEN` · `RESOLVED` · `CLOSED`.

SatQuery AI trains **six** artifacts. Four are task heads, one is a router adapter, one is a LoRA
adapter. Every backbone is **frozen** and publicly pinned by revision in `configs/base.yaml` — the
project trains small modules on top of frozen encoders, not end-to-end networks. No backbone is
fine-tuned; no backbone weight is redistributed.

> **This file is the human-readable companion to the machine-generated
> [`../models/manifest.json`](../models/manifest.json) and
> [`../models/checksums.sha256`](../models/checksums.sha256).**
> Where the two disagree, **the generated manifest wins** — it is computed by reading the files
> (`release/tools/generate_model_manifest.py`), this document is written by hand.

**The single most important rule in this document: do not fabricate.** Every number below is copied
from a file that was read, and every number names the file and (where the source is a JSON artifact)
the exact key path. Where a fact is not established, this document says so explicitly rather than
estimating.

---

## Table of contents

1. [The six trained artifacts](#1-the-six-trained-artifacts)
2. [Frozen backbones — pinned, never retrained](#2-frozen-backbones--pinned-never-retrained)
3. [Per-artifact deep reference](#3-per-artifact-deep-reference)
   - 3.0 [Enforced configuration invariants, with arithmetic](#30-enforced-configuration-invariants-with-arithmetic)
   - 3.1 [`change` — STANet-style Siamese change detector](#31-change--stanet-style-siamese-change-detector)
   - 3.2 [`change_vqa` — change question answering head](#32-change_vqa--change-question-answering-head)
   - 3.3 [`optical_sar` — CROMA-base fusion head](#33-optical_sar--croma-base-fusion-head)
   - 3.4 [`grounding` — RemoteCLIP grounding head](#34-grounding--remoteclip-grounding-head)
   - 3.5 [`router` — intent adapter over frozen MiniLM](#35-router--intent-adapter-over-frozen-minilm)
   - 3.6 [`vlm` — SmolVLM LoRA adapter (USABLE_VERIFIED, ACCEPTANCE-REJECTED)](#36-vlm--smolvlm-lora-adapter-usable_verified-acceptance-rejected)
4. [Rejected, deferred and open model decisions](#4-rejected-deferred-and-open-model-decisions)
5. [Calibration — a measured negative result](#5-calibration--a-measured-negative-result)
6. [Distribution and licensing](#6-distribution-and-licensing)
7. [Status summary — what is NOT established](#7-status-summary--what-is-not-established)
8. [Evidence index](#8-evidence-index)

---

## 1. The six trained artifacts

All six are published on the Hugging Face Hub under `thundercode/SatQuery`, one directory per task.
The table below is reproduced from [`../models/manifest.json`](../models/manifest.json)
(`artifacts[*]`), cross-checked against [`../models/checksums.sha256`](../models/checksums.sha256).

| # | `id` | Task | Artifact path | Bytes | sha256 (full) | Kind | Backbone (frozen) |
|---|---|---|---|---|---|---|---|
| 1 | `change_head` | `change` | `artifacts/change/levir_change_v001/head.pt` | 63,231,009 | `c5ef31277b67aa01a593aec0eac503eeaccc6d674349fda20ca44c9cc6f8e9fa` | trained head | none external — ResNet-18 (torchvision ImageNet) + PAM, trained in-project |
| 2 | `change_vqa_head` | `change_vqa` | `artifacts/change_vqa/run/head.pt` | 5,822,809 | `cfae5e43b97ca930f568dc5b8ae4f36b24e9ff717af226159802206ffd63a82a` | trained head | STANet change detector (frozen, backing the head's change features) |
| 3 | `optical_sar_fusion_head` | `optical_sar` | `artifacts/optical_sar/fusion_head_production_v001/head.pt` | 14,427,457 | `785815729a3a39fc34dc41894efaf00d8739365d970a3f830a326e68ae888dab` | trained head (production) | `antofuller/CROMA` (`CROMA_base.pt`, rev `0dd28e3d633b`) |
| 4 | `grounding_head` | `grounding` | `artifacts/grounding/remoteclip_grounding_v001/head.pt` | 12,639,041 | `93432f7034be91a8ffd9c1a84e3eeec00bed7832c043fe7f83d2be230284c6bb` | trained head | `chendelong/RemoteCLIP` (`RemoteCLIP-ViT-B-32.pt`, rev `bf1d8a3ccf2d`) |
| 5 | `router_adapter` | `router` | `artifacts/router/router_adapter_v001/adapter.pt` | 211,961 | `8527c3ed28a293e13293d48601d48e3ceafa137b9acabddaf5de31a58a509b5c` | trained adapter | `sentence-transformers/all-MiniLM-L6-v2` (rev `1110a243fdf4`) |
| 6 | `vlm_lora_adapter` | `vlm` | `.scratch/phase6_real_adapter/phase6_adapter/adapter_model.safetensors` | 34,798,048 | `07c76a75fa04624880ed7730590f5fdd7b145a8232e3c0af411c3c545a5adf5e` | **LoRA adapter** (PEFT) | `HuggingFaceTB/SmolVLM-500M-Instruct` (rev `a7da5b986cb5`) |

Every artifact in the manifest carries `status: "PRESENT"` and `config_hash: "78f1e3700da15aa1"`.

### 1.1 What each `kind` means

| `kind` | Meaning | How it is loaded |
|---|---|---|
| `trained_head` | A module trained in-project on top of a frozen encoder. The encoder is fetched separately. | `torch.load(...)` of a `state_dict`, or the module's own `load_*` function. |
| `trained_adapter` | A small classifier attached to a frozen sentence encoder whose embeddings are cached. | `IntentAdapter.from_config_dict` + `load_state_dict`. |
| `lora_adapter` | A PEFT LoRA delta attached to a frozen VLM at load time. | `peft.PeftModel.from_pretrained(model, dir)` (`specialists/vqa/model.py::_attach_adapter`). |

### 1.2 Training checkpoints are NOT released artifacts

Training intermediates exist on the machines that trained — e.g. the grounding head's
`checkpoint_last.pt` and the VLM adapter's `checkpoint-1500/`, `checkpoint-2000/`. These are
**archived as provenance, not released as model weights** (`docs/TRAINING.md` §8). For the VLM
adapter this matters concretely: the promoted adapter is the end-of-training **top-level** save, and
it is **not** `checkpoint-2000` (three distinct digests; see §3.6).

### 1.3 Two cross-checks where the computed hash agrees with an independently-recorded value

The manifest is generated by hashing the files on disk. For two artifacts, those computed digests can
be compared against values that were recorded *independently*, at a different time, by a different
process — which is a genuine external cross-check rather than a self-consistency claim.

**Cross-check A — `change_vqa` = `cfae5e43…`.** The manifest's `sha256` for `change_vqa/head.pt`
equals, exactly:

- `artifacts/change_vqa/run/PROMOTION.json` → `artifact.sha256`;
- the same file → `source.checkpoint_sha256_in_run_record` (the digest recorded in the Kaggle run
  record, *before* promotion);
- `artifacts/calibration_v001.json` → `provenance.checkpoint_sha256` (recorded when the temperature
  was fitted, a separate step).

`PROMOTION.json` → `source.hash_agrees_across` additionally records that the digest agrees across
`model_metadata.json`, `run_record.json` and `hashes.json (run.checkpoint_sha256)`. The bytes were
copied byte-identically (`source.byte_identical_to_source: true`) and
`artifact.weights_modified: false`.

**Cross-check B — `vlm` = `07c76a75…`.** The manifest's `sha256` for
`vlm/adapter_model.safetensors` equals, exactly:

- `artifacts/vlm/phase6_closure.json` → `production_adapter`… (via
  `artifacts/vlm/run1_test_recovery/adapter_verification.json`) →
  `weights_file_sha256` = `07c76a75fa04624880ed7730590f5fdd7b145a8232e3c0af411c3c545a5adf5e`;
- the same file → `promoted_adapter.sha256`;
- the adapter's own `ARTIFACT_SHA256SUMS.json`, against which the directory was verified
  (`manifest_check.clean: true`, `n_files_in_manifest: 14`, `n_files_on_disk: 14`,
  `mismatched_files: []`).

Both cross-checks are recorded in [`../MODEL_CARD.md`](../MODEL_CARD.md) §1 as well.

### 1.4 Why `parameters` is `null` for most artifacts

The manifest records `parameters: null` for four of the six artifacts. That is deliberate: the
generator does not open the checkpoints (opening a `.pt` to count parameters would require the model
code and torch, and would make the manifest's generation depend on the environment). Parameter counts
that *were* measured elsewhere are recorded in this document with their source. Where a parameter
count is not established, this document writes
`UNKNOWN — not established from the available evidence` rather than deriving one from file bytes.

---

## 2. Frozen backbones — pinned, never retrained

Backbones are resolved from the Hugging Face Hub on first use, pinned by revision. The revision pins
are the load-bearing part: a moving `main` would make every benchmark number unreproducible.

| Role | Repository | Revision | Size | Measured identity | Notes |
|---|---|---|---|---|---|
| Router encoder | `sentence-transformers/all-MiniLM-L6-v2` | `1110a243fdf4` | 90.9 MB | 22,713,216 params, 384-dim embeddings | tokenizer ceiling **256**; truncation set to **128** |
| VLM | `HuggingFaceTB/SmolVLM-500M-Instruct` | `a7da5b986cb5` | ~1015 MB safetensors | 516,165,824 params (base) | processor `longest_edge` **must** be pinned (F5-2) |
| Grounding | `chendelong/RemoteCLIP` (`RemoteCLIP-ViT-B-32.pt`) | `bf1d8a3ccf2d` | 605.2 MB | 151,277,313 params; width **768**, **projected** dim **512** | patch size 32; 7×7 tokens at 224 |
| Optical-SAR | `antofuller/CROMA` (`CROMA_base.pt`) | `0dd28e3d633b` | 777.6 MB (777,563,846 bytes) | 194,365,440 params; `encoder_dim` 768 | `image_resolution` 120; asymmetric `s1_depth=6`, `s2_depth=12` |
| Change encoder | — (torchvision) | — | — | ResNet-18, `IMAGENET1K_V1` | `pretrained_used: true` in the trained artifact |

### 2.1 Router encoder — `all-MiniLM-L6-v2`

- Declared in `configs/base.yaml` under `router:` — `model`, `revision: 1110a243fdf4`,
  `max_length: 128`, `embedding_dim: 384`.
- **Finding F4-1.** The MiniLM tokenizer's own ceiling is **256** (verified by probe). `128` is a
  deliberate truncation well inside that ceiling, not the model limit. The encoder **asserts the
  configured value is ≤ 256**, because truncating above the ceiling is a silent no-op — a control that
  appears to work and does nothing.
- **Finding F4-2.** The encoder is frozen, so embeddings are cached and the adapter trains on cached
  vectors. Measured on CPU: **20 epochs over 4,096 × 384 in 0.28 s**. No GPU required.
- The revision is asserted by `router/encoder.py` (guards on revision and `max_length`).

### 2.2 VLM — `SmolVLM-500M-Instruct`

- Declared under `vlm:` — `checkpoint`, `revision: a7da5b986cb5`, `max_new_tokens: 128`,
  `do_sample: false`, `temperature: 0.0`, `processor_longest_edge: 512`, `do_image_splitting: true`,
  `max_images_per_call: 1`.
- **Finding F5-1.** `AutoModelForVision2Seq` does **not exist** in transformers 5.17.0 (it is not
  merely deprecated — referencing it raises `AttributeError`). The loader class is resolved by
  feature detection over `("AutoModelForImageTextToText", "AutoModelForVision2Seq",
  "AutoModelForMultimodalLM")`, never hardcoded (`specialists/vqa/model.py::resolve_loader_class`).
- **Finding F5-2 (cost).** The processor's default `longest_edge` is 2048, which upscales a 512-px
  tile 4× and then splits it (`do_image_splitting=True`) into **17 sub-images** and 1142 prompt
  tokens. MEASURED: default → `pixel_values (1, 17, 3, 512, 512)`; pinned → `pixel_values
  (1, 1, 3, 512, 512)`. The plan estimated a 4× cost overrun; the real figure is **~17×**. The pin
  **must** be set explicitly on the processor at construction time, and `generate` raises loudly if
  more than one image is produced for a single input.
- **Finding F5-3.** SmolVLM requires one `<image>` token per image in the prompt; hand-written prompt
  strings raise `ValueError`. All prompts are built through `processor.apply_chat_template()`.
- **Finding F5-4.** The dtype kwarg is not discoverable by signature (`from_pretrained` is
  `**kwargs`-only). Resolved by a call-time fallback over `("dtype", "torch_dtype")`.

### 2.3 Grounding — `RemoteCLIP ViT-B/32`

- Declared under `grounding:` — `checkpoint_repo`, `checkpoint_file: RemoteCLIP-ViT-B-32.pt`,
  `checkpoint_revision: bf1d8a3ccf2d`, `model_name: ViT-B-32`, `image_size: 224`,
  `resolution_frozen: true`, `nms_iou: 0.50`, `max_candidates: 20`, `confidence_threshold: 0.40`,
  `benchmark_box_scale: 100.0`, `coordinate_system: normalized_0_1`,
  `encoder_projected_dim: 512`.
- **Measured, not assumed** (`specialists/grounding/remoteclip.py`): `VERIFIED_PATCH_SIZE = 32`,
  `VERIFIED_TRANSFORMER_WIDTH = 768`, `VERIFIED_PROJECTED_DIM = 512`,
  `VERIFIED_PARAMETERS = 151_277_313`. `_verify_contract()` asserts all four against the loaded model
  at load time.
- **The 768-vs-512 distinction is the one that bites.** `visual.positional_embedding` is 768 wide and
  `visual.proj` is `(768, 512)`; the embeddings the text tower can be compared against are the
  **projected** ones (512). Using 768 anywhere here is a shape error torch would surface only at the
  similarity step — after the patch features have already been computed and cached.
- Tokens: 7×7 = 49 (+1 CLS) at 224; 14×14 = 196 (+1 CLS) at 448. `SUPPORTED_RESOLUTIONS = (224, 448)`;
  which one to *use* was an empirical question answered by the resolution experiment (§4.1), and
  neither is marked "preferred" in the encoder.

### 2.4 Optical-SAR — `CROMA-base`

- Declared under `croma:` — `checkpoint_repo`, `checkpoint_file: CROMA_base.pt`,
  `checkpoint_revision: 0dd28e3d633b`, `variant: base`, `image_resolution: 120`, `encoder_dim: 768`,
  `optical_channels: 12`, `sar_channels: 2`, `modalities: [optical, sar, joint]`,
  `modalities_used: [optical, sar, joint]`.
- **Finding C-7.** `image_resolution % 8 == 0`; native 120 → 15×15 = **225 patches**. Enforced in
  `core/config.py` and re-checked in `CROMAEncoder.__init__`.
- **Finding C-1 — CROMA is never given a mask.** The forward pass takes exactly two arguments,
  `model(SAR_images=..., optical_images=...)`. The availability mask is consumed by the **fusion
  head**, not by CROMA (see §3.3). The rationale: CROMA is a masked autoencoder, and handing it an
  availability mask invites it to reconstruct missing channels — precisely the fabrication the sensor
  adapter exists to prevent.
- First real forward pass (measured): CPU, 0.89 s for a batch of 2 at 120 px, 194,365,440 parameters,
  `n_patches = 225`, `(B, 768)` GAPs and `(B, 225, 768)` tokens confirmed. Two facts no document had
  recorded surfaced: CROMA-base is **asymmetric** (`s1_depth=6`, `s2_depth=12`) and the joint
  cross-attention is **directional** (SAR queries optical).
- **DEV-1 (constructor).** CROMA is distributed as a GitHub repository, not a pip package; the
  constructor is `use_croma.PretrainedCROMA`, and `use_croma.py` **must be vendored**. Ruled
  **ACCEPTED as an implementation detail** — no `ARCHITECTURE CHANGE` entry
  (`docs/PHASE14_OPTICAL_SAR_DECISIONS.md` §2). The vendored file is recorded at
  `specialists/optical_sar/vendor/use_croma.py`, 14,556 bytes, sha256
  `a38567beed29eb08108a47cdc97fe98aec50fd4be0bd98a5266bcd18aafb7c5b`.
- **DEV-2 (input normalisation).** Resolved 2026-09-18: option (a) — match upstream's per-channel
  `mean ± 2·std` stretch immediately before the forward pass with `use_8_bit` enabled — adopted as the
  required implementation; option (c) registered as a **gated** experiment with (a) as its control;
  option (b) rejected as a standalone path. Implemented in `specialists/optical_sar/radiometry.py`
  and wired through `croma.py`.

### 2.5 Change encoder — ResNet-18, trained in-project

The change detector's encoder is **not** an external pinned backbone in the sense the others are: it
is a torchvision `resnet18`, requested with `change.pretrained: true` and recorded in the trained
artifact's embedded config as `pretrained_used: true` with `encoder: "resnet18"` and
`encoder_channels: [64, 128, 256, 512]`.

The code does **not** make pretrained weights a hard dependency. Resolution order
(`specialists/change/stanet.py`): (1) a local weights file, if given; (2) torchvision's
`IMAGENET1K_V1` download; (3) random initialisation **with an explicit warning** — because
`pretrained: true` in a config must not become a lie when a download fails. The trained artifact
records `pretrained_used: true`, so this run used ImageNet weights.

### 2.6 Backbone licensing

Backbones are **not redistributed** by this project. They are fetched from the Hub at run time and
carry their own licences (see each model's HF page). See §6.

---

## 3. Per-artifact deep reference

Each subsection below gives, for one artifact: its path, byte count, sha256, architecture, **every
hyperparameter** (from `configs/base.yaml` and the code), the backbone and its pinned revision, the
training data, the evaluation protocol, the measured numbers (with artifact key paths), the
acceptance status, and its limitations.

### 3.0 Enforced configuration invariants, with arithmetic

Two invariants are **enforced at load time**, not merely documented. Both exist because the failure
mode they prevent is a *silent shape error* — a tensor of exactly the right shape that trains to a
worse number, the hardest kind of bug to notice.

#### 3.0.1 `fusion.input_dim == 2318`

```
input_dim = len(modalities_used) * encoder_dim + optical_channels + sar_channels
          = 3 * 768              + 12             + 2
          = 2304                 + 14
          = 2318
```

`configs/base.yaml` states the arithmetic in a comment above `fusion.input_dim: 2318`, and the code
recomputes it **twice**, independently:

- `core/config.py` recomputes it as
  `len(modalities_used) * encoder_dim + optical_channels + sar_channels` at load time (finding C-1);
- `specialists/optical_sar/fusion_head.py::expected_fusion_dim` returns
  `int(modalities_used) * int(encoder_dim) + int(optical_channels) + int(sar_channels)`;
- `build_fusion_head` **raises `ModelLoadError`** if `int(input_dim) != 2318`, naming the arithmetic
  (`3*768 + 12 + 2`) in the message;
- `assemble_fusion_input` concatenates `[optical_GAP, sar_GAP, joint_GAP, optical_mask, sar_mask]`
  and **raises `SpecialistError`** if the assembled width is not the expected width.

The order of the concatenation is frozen (freeze §2.5) and **asserted rather than assumed**, because a
permutation produces a tensor of exactly the right shape that trains to a worse number.

> The mask goes **here**, not into CROMA. Freeze §2.5: channel/band dropout during fusion-head
> training is **mandatory** — it is what teaches the head to trust the availability mask. With no
> dropout the head learns to read channel 4 unconditionally, because in training channel 4 was always
> present; on a 4-band sensor channels 5–12 are *always* zero, and a head that never saw a masked
> channel treats those zeros as a measurement of blackness rather than as absence.
> `channel_dropout()` is a real function, tested directly (measured: `p=1.0` keeps 48/48; `p=0.4`
> drops to 18/48; dropped features are exactly `0.0`, not renormalised; a band the sensor never
> measured stays dropped even at `p=1.0`; the seeded RNG is deterministic).

#### 3.0.2 `grounding_head.feature_dim == 2048`

```
feature_dim = 4 * grounding.encoder_projected_dim
            = 4 * 512
            = 2048
```

The per-cell feature is

```
f_i = concat([p_i, t, p_i * t, global_pool])        -> 4 * 512 = 2048
```

where `p_i` is the patch token, `t` the text embedding broadcast to every cell, `p_i * t` the
element-wise cross-modal alignment, and `global_pool` the mean over all patches. The global term
matters: a per-cell MLP otherwise cannot see anything outside its own patch, and a 1/7-of-image
receptive field is too small for objects that span several cells.

The invariant is enforced **three** ways:

- `core/config.py` **rejects any value other than** `4 * grounding.encoder_projected_dim` at load
  time — deliberately, so the head can be validated *without importing torch*;
- `specialists/grounding/head.py::build_head` recomputes `expected = 4 * VERIFIED_PROJECTED_DIM` and
  raises `ValueError` naming both numbers if they differ;
- `GroundingHead.forward` re-checks `feat.shape[-1] != self.feature_dim` and raises, and
  `RemoteCLIPEncoder._verify_contract` asserts `visual.proj.shape[1] == 512` against the real model.

A mismatch is a silent shape error: torch raises only at the similarity step, after the patch
features are already cached.

#### 3.0.3 Other enforced invariants (summary)

| Invariant | Enforced where | Failure mode prevented |
|---|---|---|
| `croma.image_resolution % 8 == 0` | `core/config.py`, `CROMAEncoder.__init__`, `OpticalSarSpecialist.__init__` | CROMA asserts it internally (finding C-7) |
| `router.max_length <= 256` | `router/encoder.py` | truncating above the tokenizer ceiling is a silent no-op |
| `change` spatial dims divisible by 8 | `STANetStyleChangeDetector.forward` | encoder stride; a 1-px-off raster is reflect-padded and cropped back, with a warning |
| `sa_mode in {PAM, BAM}`; `BAM` raises | `STANetStyleChangeDetector.__init__` / `.forward` | a config asking for BAM fails visibly rather than silently aliasing PAM |
| `fusion_head` device must be `"cpu"` | `build_fusion_head` | a device string that would silently fall back is worse than a refusal |
| `task_dim >= 2` | `OpticalSarSpecialist.__init__` | a margin cannot exist over one class |
| exactly 2 assets for change / change_vqa / optical_sar; exactly 1 for vqa/caption | each specialist's `validate_request` | a silent "just use the first two" produces a confident answer to a question the caller did not ask |

---

### 3.1 `change` — STANet-style Siamese change detector

**Status:** `IMPLEMENTED`, `VERIFIED` — the only task whose headline metric carries the `VERIFIED`
tag, because it is the only one measured against a single, immutable public test split with a frozen
threshold.

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `change_head` | `models/manifest.json` |
| Path | `artifacts/change/levir_change_v001/head.pt` | manifest |
| Bytes | 63,231,009 | manifest |
| sha256 | `c5ef31277b67aa01a593aec0eac503eeaccc6d674349fda20ca44c9cc6f8e9fa` | manifest, `checksums.sha256` |
| HF path | `change/head.pt` | manifest |
| Architecture | STANet-style Siamese change detector (ResNet-18 + PAM) | manifest, code |
| `kind` | `trained_head` | manifest |
| `backbone` | `null` (trained in-project; ResNet-18 ImageNet trunk) | manifest |
| Config hash | `78f1e3700da15aa1` | manifest |
| Parameters | `UNKNOWN — not established from the available evidence` (manifest records `null`) | manifest |

#### Architecture (`specialists/change/stanet.py`)

A STANet-shaped Siamese network — shared Siamese encoder, spatial-temporal attention over feature
differences, feature-difference aggregation decoder — **reimplemented rather than vendored**, per the
Phase 0 resolution of finding C-9 (the upstream repo is Python 3.6-era and depends on `visdom`/`apex`).

```
T1 ---> [ shared encoder ] ---> f1  (4 levels)
T2 ---> [ shared encoder ] ---> f2  (4 levels)
                                  |
                      |f1 - f2| + concat([f1, f2, |f1-f2|])
                                  |
                          spatial attention (PAM)
                                  |
                        progressive decoder + skips
                                  |
                          1-channel change logit
```

- **Weights are tied, not copied.** Both branches call the **same** module instance
  (`SharedResNetEncoder`), so there is no second encoder to fall out of sync — the "shared encoder"
  requirement from plan §16 enforced by construction, not by convention.
- **Measured encoder shapes** (ResNet-18 trunk, 256×256 input):

  | Level | Channels | Grid | Positions |
  |---|---|---|---|
  | `layer1` | 64 | 64×64 | 4096 |
  | `layer2` | 128 | 32×32 | 1024 |
  | `layer3` | 256 | 16×16 | 256 |
  | `layer4` | 512 | 8×8 | 64 |

- **The attention memory line is computed, not assumed.** STANet's PAM builds a full
  `positions × positions` attention matrix. At batch 8, fp32:

  ```
  layer1   8 * 4096^2 * 4 =  537.0 MB   <- exceeds a Kaggle session's budget
  layer2   8 * 1024^2 * 4 =   33.5 MB   <- fine
  layer3   8 *  256^2 * 4 =    2.1 MB   <- fine
  layer4   8 *   64^2 * 4 =    0.13 MB  <- fine
  ```

  Attention is therefore applied at **layers 2, 3 and 4** and **skipped at layer 1**, where the
  difference features are fused directly instead. The decision is **recomputed on every forward pass**
  from the actual batch size and a byte budget (`DEFAULT_ATTENTION_BUDGET_BYTES = 256 * 1024 * 1024`),
  so changing batch size or tile size moves the line correctly rather than silently overrunning
  memory. Applied/skipped levels are returned in the output so the trace can report them
  (`ChangeOutput.to_trace`). **This is a deviation from a literal STANet reproduction, recorded as
  one** — attention at 64×64 would not fit. It is not a silent substitution.

- **Difference fusion.** `DifferenceFusion` reduces `concat([f1, f2, |f1-f2|])` (3C channels) back to
  a working width with `Conv2d(3C, out, 3, padding=1, bias=False) → BatchNorm2d → ReLU`.
- **Attention module.** `SpatialAttention` is PAM-style: 1×1 `query`/`key` convolutions to
  `hidden = max(1, channels // reduction)` with `reduction = 8`, a 1×1 `value` to `channels`, and a
  1×1 `out`; `softmax(q @ k / sqrt(hidden))` over positions, residual `x + out(attended)`.
- **Decoder.** Progressive, coarse-to-fine: `dec3` and `dec2` are `DecoderBlock(width, width, width)`;
  `dec1` is `DecoderBlock(width, width, width // 2)`; `final_up` is
  `ConvTranspose2d(width//2, width//2, 4, stride=4)`; `head` is `Conv2d(width//2, 1, 1)` with
  `FINAL_BIAS_INIT = -2.0` (LEVIR-CD's changed-pixel fraction is roughly 5–15%, so a bias of −2.0
  starts the prior near 0.12 rather than 0.5 and stops the first epochs being spent un-learning a
  saturated sigmoid).
- **Loss.** Composite BCE + Dice, `change_loss(logits, target, bce_weight=0.5, dice_weight=0.5,
  pos_weight=None)`. Dice is computed on soft probabilities so it is differentiable and directly
  optimises region overlap; `eps = 1e-6` guards the empty-target case (a tile with no change would
  otherwise be `0/0 = NaN`). `pos_weight` is left as an explicit knob rather than a hidden default,
  because Dice already addresses the class imbalance.

#### Hyperparameters — every value

From `configs/base.yaml` (`change:`):

| Key | Value |
|---|---|
| `tile_size` | 256 |
| `tile_overlap` | 0 |
| `threshold` | 0.50 |
| `min_component_pixels` | 32 |
| `encoder` | `resnet18` |
| `sa_mode` | `PAM` (`BAM` declared but not implemented; raises) |
| `pretrained` | true |
| `learning_rate` | 0.001 |
| `batch_size` | 8 |
| `bce_weight` | 0.5 |
| `dice_weight` | 0.5 |
| `levir_split.train` | 7120 |
| `levir_split.val` | 1024 |
| `levir_split.test` | 2048 |

From the trained artifact's **embedded** config
(`artifacts/change/eval_test/eval_result.json` → `checkpoint_embedded_config`), which is the config
actually baked into the weights:

| Key | Value |
|---|---|
| `width` | 128 |
| `sa_mode` | `PAM` |
| `attention_budget_bytes` | 268435456 (256 MB) |
| `encoder` | `resnet18` |
| `encoder_channels` | `[64, 128, 256, 512]` |
| `frozen_encoder` | false |
| `pretrained_used` | true |

#### Training data

LEVIR-CD-256 — the standard 256×256 change-detection benchmark. Split used (from `configs/base.yaml`):
train **7,120** / val **1,024** / test **2,048**. The test split is immutable and public.

#### Evaluation protocol

`artifacts/change/eval_test/eval_result.json`. Environment recorded in the artifact: device `cuda`,
torch `2.10.0+cu128`, Python `3.12.13`, platform `Linux-6.12.90+-x86_64-with-glibc2.35`,
`cuda_available: true`, `seconds: 55.359`. Config-hash integrity is recorded explicitly:
`config_hash: 78f1e3700da15aa1`, `checkpoint_config_hash: 78f1e3700da15aa1`,
`checkpoint_config_hash_checked: true`, `config_drift: false`,
`config_drift_acknowledged: false`.

Evaluation population: `n: 2048`, `n_images_with_change: 935`,
`mean_change_fraction: 0.0509`, `n_pixels: 134217728`, `threshold: 0.5`.

Change-fraction quantiles (`change_fraction_quantiles`): `min 0.0`, `p50 0.0`, `p90 0.197205`,
`max 0.684937`. The median tile has **zero** changed pixels, which is why pooled and macro metrics are
both reported.

#### Measured numbers

All from `artifacts/change/eval_test/eval_result.json`.

| Metric | Value | Exact key path |
|---|---|---|
| **pooled IoU** | **0.8122** | `metrics.pooled.iou` |
| **macro IoU** | **0.8457** | `metrics.macro.miou` |
| **pooled F1** | **0.8964** | `metrics.pooled.f1` |
| pooled mean IoU (`miou`) | 0.9007 | `metrics.pooled.miou` |
| pooled precision | 0.9195 | `metrics.pooled.precision` |
| pooled recall | 0.8745 | `metrics.pooled.recall` |
| macro IoU (`iou`, per-class change IoU) | 0.718 | `metrics.macro.iou` |
| macro F1 | 0.7962 | `metrics.macro.f1` |
| macro precision | 0.8506 | `metrics.macro.precision` |
| macro recall | 0.7757 | `metrics.macro.recall` |
| tp | 5,978,997 | `metrics.pooled.tp` |
| fp | 523,658 | `metrics.pooled.fp` |
| fn | 858,407 | `metrics.pooled.fn` |
| tn | 126,856,666 | `metrics.pooled.tn` |

The confusion counts are identical between the pooled and macro blocks (as they must be — they are the
same pixel population; the two blocks differ only in how the class-wise scores are aggregated).

**Pooled vs macro, and why both.** The test split is only ≈ 5 % changed pixels. Pooled IoU answers
"how well does the mask overlap overall"; macro IoU answers "how well does each class do, averaged".
Quoting one alone would hide the imbalance question.

#### Acceptance status

`VERIFIED`. This is the only task whose headline number carries the `VERIFIED` tag in
[`BENCHMARKS.md`](BENCHMARKS.md) §1, because it is the only one measured against a single, immutable
public test split with a frozen threshold.

#### Serving wiring (and why the checkpoint is not in the config)

A trained head exists and is benchmarked, yet it is **deliberately not wired into serving by default**
(`specialists/change/specialist.py`). Populating `change.checkpoint_path` in `configs/base.yaml` would
move `Config.hash`, and `scripts/eval_change.py` refuses to score on a hash drift (exit 3) — so that
one edit would invalidate the project's own benchmark number. The wiring path is therefore the
registry `builders=` override (`core/registry.py`), which injects the checkpoint without touching the
config. With no checkpoint wired, a missing artifact is a **deployment** case, not a crash: the
specialist runs the randomly-initialised model, marks the result `degraded`, and says so plainly,
because a randomly-initialised change map is a map of noise and presenting it as a detection would be
exactly the fabrication the evidence system exists to prevent. `has_checkpoint` distinguishes "a
checkpoint was loaded" from "we built a random one and are being honest about it".

#### Limitations

- **Absolute IoU is high, but this is one benchmark.** LEVIR-CD-256 is overhead optical; it says
  nothing about the hidden ISRO/SAC distribution (Cartosat-2S + RISAT).
- **Attention is skipped at layer 1** — a recorded deviation from literal STANet, forced by memory.
- **The detector is not the product path by default** (§ above).
- **Post-processing suppresses spatial claims on a mis-registered pair.** Poor co-registration drives
  confidence to 0.0 and withholds region claims; the maps are still written, but the region assertions
  are not made. This is by design (`architecture freeze §2.4`, plan §40) but it means the reported
  metric is for a well-registered benchmark, not for arbitrary pairs.
- **Minimum region size is 32 px** (`change.min_component_pixels`), so sub-32-px changes are
  discarded by construction.

---

### 3.2 `change_vqa` — change question answering head

**Status:** `IMPLEMENTED`, `MEASURED`, ruling **OPEN**.

A head that answers natural-language change questions over a temporal pair. It answers the eight CDVQA
question types (`change_or_not`, `change_ratio`, `change_ratio_types`, `change_to_what`,
`increase_or_not`, `decrease_or_not`, `largest_change`, `smallest_change`) over a **closed 19-answer
space**.

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `change_vqa_head` | `models/manifest.json` |
| Path | `artifacts/change_vqa/run/head.pt` | manifest |
| Bytes | 5,822,809 | manifest |
| sha256 | `cfae5e43b97ca930f568dc5b8ae4f36b24e9ff717af226159802206ffd63a82a` | manifest, `PROMOTION.json` |
| HF path | `change_vqa/head.pt` | manifest |
| Architecture | `change_vqa_head_v1` | `PROMOTION.json` → `artifact.architecture` |
| **Parameters** | **1,453,912** | `PROMOTION.json` → `artifact.parameters` |
| Backbone | STANet change detector (frozen, backing the head's change features) | manifest |
| Config hash | `78f1e3700da15aa1` | manifest |
| `satquery_trained` | true | `PROMOTION.json` → `artifact.satquery_trained` |
| `eval_mode` | true | `PROMOTION.json` → `artifact.eval_mode` |
| `non_finite_tensors` | **0** | `PROMOTION.json` → `artifact.non_finite_tensors` |
| `weights_modified` | **false** | `PROMOTION.json` → `artifact.weights_modified` |

#### Architecture and the frozen dependency

The head consumes **cached features**, not raw imagery: a frozen STANet detector produces a change
representation, and a frozen question text encoder (`sentence-transformers/all-MiniLM-L6-v2`) produces
a text vector; the head maps those to an answer index. Recorded feature specs
(`PROMOTION.json` → `identity`): `feature_spec: change_feat_v1`,
`change_cache_spec: c801326f85a185f8`, `text_cache_spec: d2801ea1a314354a`,
`preprocessing_version: change_vqa_preproc_v1`.

The **frozen dependency** is the STANet detector itself (`PROMOTION.json` → `frozen_dependency`):

| Field | Value |
|---|---|
| role | STANet change detector backing the head's change features |
| path | `artifacts/change/levir_change_v001/head.pt` |
| sha256 | `c5ef31277b67aa01a593aec0eac503eeaccc6d674349fda20ca44c9cc6f8e9fa` |
| bytes | 63,231,009 |
| `detector_trained` | true |
| `verified_byte_exact_vs_local` | true |

This digest is byte-identical to the `change` artifact in §3.1 — the head is wired to the same
checkpoint the change task releases.

**The detector is frozen and shared in spirit, not in instance** (`specialists/change/vqa_specialist.py`).
When the planner routes a change+language request it plans **both** a `change` step and a
`change_vqa` step, so the detector runs twice for one request. That is a real cost, stated rather than
hidden: the weights are loaded once (the registry caches the specialist instance), so the second cost
is a forward pass, not a 60 MB load. The alternative (passing the change map between steps) needs the
controller to hand artifacts between steps, which it does not do today. `execute` therefore accepts an
optional `change_map` path in `request.params` for a future planner to populate — **unused today and
documented as such**, so the hook exists without pretending the wiring does.

#### Hyperparameters and selection

| Key | Value | Source |
|---|---|---|
| `apply_type_mask` | true (serving default) | config + `vqa_specialist.py` |
| `image_size` | 256 (feature extractor default) | `vqa_specialist.py` (`change_vqa.image_size`) |
| `seed` | 42 | `PROMOTION.json` → `identity.seed` |
| `epoch_selected` | **8** | `PROMOTION.json` → `identity.epoch_selected` |
| `selected_on` | Val answer accuracy | `PROMOTION.json` → `identity.selected_on` |
| `val_answer_accuracy` | **0.700018** | `PROMOTION.json` → `identity.val_answer_accuracy` |
| `stop_reason` | `early_stopping` | `PROMOTION.json` → `identity.stop_reason` |
| `LOW_CONFIDENCE_THRESHOLD` | 0.40 (a reporting threshold, not a calibration) | `vqa_specialist.py` |

The **type mask** restricts the answer to those legal for the question type. It is on by default in
serving because the type is known from the question and an illegal answer is never right; the
evaluation reports masked and unmasked separately so the mask's contribution is visible.
`PROMOTION.json` → `verification.mask_gain` is **0.0**.

#### Training data

CDVQA annotations + SECOND (SCD) imagery (`docs/DATASETS.md` §3). The CDVQA repository publishes
**annotations only**; imagery is acquired from SECOND, name-verified **2,968 / 2,968 MATCH**, extracted
to `data/cdvqa/{im1,im2,label1,label2}/`. Val images 400; val questions **16,441**; test questions
**39,686**.

**Temporal and label semantics** — established from evidence, with honest uncertainty
(`vqa_specialist.py::TEMPORAL_ORDER_NOTE`): `T1 = pre, T2 = post`; `label1=pre, label2=post` is
**proven** (agreement 1.0000 over 2,968 scenes); `im1=pre, im2=post` is **supported statistically, not
proven**. The distinction travels in the evidence payload so a reader can see which leg the result
rests on.

**Trained externally.** The head was trained outside the repository, on an external GPU (Kaggle),
following `docs/R02_KAGGLE_TRAINING_GUIDE.md`, whose entry status was
`IMPLEMENTATION_READY_FOR_EXTERNAL_TRAINING` and whose explicit contract is: *training produces an
artifact, not a verified capability, and the run record says `TRAINED_UNVERIFIED`.* The returned
checkpoint was then **promoted** through a byte-identity gate
(`PROMOTION.json` → `state`): `before_promotion: TRAINED_UNVERIFIED`,
`after_promotion: PROMOTED`. The note is explicit: *promotion records provenance and wires the serving
path; it does not itself confer VERIFIED status — that is the maintainer's ruling.*

#### Evaluation protocol

`artifacts/change_vqa/run/PROMOTION.json` → `verification`. Verification report
`ARTIFACT_VERIFICATION.md` (2026-09-22): `checks_passed: 93`, `checks_failed: 0`,
`checks_unverified: 0`.

**Two test sets are reported. Quoting only the better one would be selective.**

| Test set | n | accuracy | macro F1 |
|---|---|---|---|
| `test` | 39,686 | **0.697626** | **0.378373** |
| `test2` | 31,036 | **0.651469** | **0.372309** |

| Field | Value | Exact key path |
|---|---|---|
| test accuracy | 0.697626367 | `verification.test_accuracy` |
| test macro F1 | 0.378373275 | `verification.test_macro_f1` |
| test2 accuracy | 0.651469262 | `verification.test2_accuracy` |
| test2 macro F1 | 0.372308516 | `verification.test2_macro_f1` |
| n scored, test | 39,686 | `verification.n_scored_test` |
| n scored, test2 | 31,036 | `verification.n_scored_test2` |
| global majority baseline, test | 0.311546 | `verification.global_majority_baseline_test` |
| global majority baseline, test2 | 0.178728 | `verification.global_majority_baseline_test2` |
| mask gain | 0.0 | `verification.mask_gain` |
| metric ruling | `OPEN — the plan leaves the accuracy/macro-F1 interpretation owner-gated. No official aggregate metric is asserted here.` | `verification.metric_ruling` |

**The wide gap between accuracy and macro-F1 means the head is carried by common classes and performs
poorly on rare ones.** The global-majority baselines make the gap concrete: 0.311546 on `test` means a
constant predictor would score 0.31, and the head scores 0.70; but the macro-F1 of 0.378 shows the
per-class picture is far weaker than the aggregate.

#### Acceptance status

**OPEN.** No promotion/acceptance decision has been recorded. `PROMOTION.json` → `state.note` states
this explicitly. The `metric_ruling` is `OPEN`.

#### Serving behaviour — degraded mode produces no answer, on purpose

`ChangeSpecialist` degrades to an untrained detector and still emits a change map. This specialist must
**not** copy that, because the outputs are not comparable: a random change map is visibly noise,
whereas an untrained 19-way classifier still emits a fluent, confident-looking `yes`. A user cannot
tell the second from a real answer, so the untrained case returns **no answer at all** — `answer=""`,
`degraded=True`, and a warning naming exactly which piece is missing. The controller then falls through
to another attributed specialist.

There is also a **feature-spec mismatch refusal**: `feature_spec_mismatch()` compares the spec hash the
head was trained on against the spec hash this deployment produces, and refuses to answer if they
differ. The reason is measured, not hypothetical: `configs/base.yaml`'s `change:` section carries no
`checkpoint_path`, so the registry builds the change feature extractor with `checkpoint_path=None` and
gets an **untrained** STANet — while `scripts/prepare_change_vqa.py` defaults to the trained LEVIR
checkpoint. Training and serving would consume different representations, and the head would still emit
a fluent `yes`. So it refuses, and names both specs.

Confidence is **uncalibrated and says so**: `ConfidenceBreakdown.method` is `"uncalibrated"`, `raw` is
the softmax probability of the chosen answer, `components` carries the top1-top2 margin, the number of
distinct answers seen, and the normalised entropy. Nothing is fitted here — the calibration contract
(§5) is a separate artefact.

#### Limitations

- **Weak on rare classes** (macro-F1 0.378 / 0.372 vs accuracy 0.698 / 0.651).
- **Two test sets, both reported** — and the second is materially lower on accuracy (0.651 vs 0.698).
- **Ruling OPEN** — no official aggregate metric is asserted.
- **Serving needs two artefacts to be honest**: a trained head *and* a trained feature extractor; the
  shipped config wires neither by default.
- **`im1=pre, im2=post` is supported statistically but not proven** — a residual provenance risk.

---

### 3.3 `optical_sar` — CROMA-base fusion head

**Status:** `IMPLEMENTED`, `MEASURED`, ruling **OPEN**. This is the spec's #1 evaluation priority, and
it is the only workflow whose inputs are two **different sensors**.

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `optical_sar_fusion_head` | `models/manifest.json` |
| Path | `artifacts/optical_sar/fusion_head_production_v001/head.pt` | manifest |
| Bytes | 14,427,457 | manifest, `pre_registered_115_metric.json` → `head_bytes` |
| sha256 | `785815729a3a39fc34dc41894efaf00d8739365d970a3f830a326e68ae888dab` | manifest, `checksums.sha256`, metric JSON `head_sha256` |
| HF path | `optical_sar/head.pt` | manifest |
| Architecture | CROMA-base fusion head (input_dim 2318 → hidden 512 → 19 classes) | manifest |
| Backbone | `antofuller/CROMA` (`CROMA_base.pt`, rev `0dd28e3d633b`) | manifest |
| Config hash | `78f1e3700da15aa1` | manifest |
| Parameters | `UNKNOWN — not established from the available evidence` (manifest records `null`) | manifest |

#### Architecture (`specialists/optical_sar/fusion_head.py`)

CROMA produces three 768-dimensional vectors per sample (an optical GAP, a SAR GAP, and a joint GAP).
The fusion head turns those, plus the availability masks, into a task prediction.

```
    optical_GAP      (B, 768)
    SAR_GAP          (B, 768)
    joint_GAP        (B, 768)
    optical_mask     (B,  12)     <- availability, from the sensor adapter
    sar_mask         (B,   2)     <- availability, from the sensor adapter
                     ---------
    concat           (B, 2318)
```

The frozen head architecture (`build_fusion_head`, freeze §2.5 / plan §17):

```
LayerNorm -> Linear(input_dim, hidden_dim) -> GELU -> Dropout -> Linear(hidden_dim, task_dim)
```

No activation on the output: this is a logit-producing task head, and a softmax here would be applied
twice once a loss function adds its own.

**The mask goes here, not into CROMA (finding C-1).** See §3.0.1. `CROMA_GAP_KEYS = ("optical_GAP",
"SAR_GAP", "joint_GAP")` is declared once so the rest of the package refers to one spelling.

#### Hyperparameters — every value

From `configs/base.yaml` (`croma:` and `fusion:`):

| Key | Value |
|---|---|
| `croma.checkpoint_repo` | `antofuller/CROMA` |
| `croma.checkpoint_file` | `CROMA_base.pt` |
| `croma.checkpoint_revision` | `0dd28e3d633b` |
| `croma.variant` | `base` |
| `croma.image_resolution` | 120 |
| `croma.encoder_dim` | 768 |
| `croma.optical_channels` | 12 |
| `croma.sar_channels` | 2 |
| `croma.modalities` | `[optical, sar, joint]` |
| `croma.modalities_used` | `[optical, sar, joint]` |
| `fusion.input_dim` | 2318 |
| `fusion.hidden_dim` | 512 |
| `fusion.dropout` | 0.2 |
| `fusion.num_classes` | 19 |

**Seed sweep.** Training was run as two arms (**armA**, **armB**) × five seeds (**100–104**), with a
per-arm variance report (`armA_seed_variance_report.json`). The **production head** is a distinct,
frozen artifact (`fusion_head_production_v001/head.pt`) with its own `production_head_record.json` and
a `phase12_rerun_verification.json`.

**A/B arm decision.** Made separately on `best_val_accuracy`: **A 0.837100 vs B 0.839100**, floor
**0.0285** → **Arm A retained** (owner ruling R-14,
`docs/PHASE12_115_METRIC_COMPUTED.md` §5). The metric JSON records `cache_arm: "A"`.

**Channel dropout is mandatory** (freeze §2.5). Plan §20's robustness schedule: optical is trained at
100/80/60/40 % channel availability and SAR at 100/50 %. The mask is updated to match the dropped
features — that is the entire point: dropping features while leaving the mask saying "present" would
teach the head that the mask lies. Dropped channels are zeroed, **not renormalised** — renormalising
would fabricate a scale the real missing-channel case does not have.

#### Training data

BigEarthNet (CLC-19), **single-label subset** (see the caveat below). Measured local subset: 28,000
S2 patches, 98 tiles, 12 bands per patch; full official corpus 480,038 patches. Training consumes
cached CROMA features (`fusion_features/`, `fusion_features_armB/`, ~231 MB each), which are
**reproducible and not released as model weights**.

**Measured context for the single-label policy:** single-label patches are **17.57 %** of the corpus
(96,537 of 549,488), and under this policy the rarest class survives as **1 patch** — a **59,204 : 1**
imbalance (`docs/PHASE12_LABEL_POLICY_DECISION.md` §3.0).

#### Evaluation protocol

`artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json`. This is a
**separate, later, read-only** step (`scripts/eval_fusion_115.py`), deliberately **not** part of the
trainer: every `run_record.json` carries `pre_registered_metric_computed = false` and
`result_status = "PLUMBING_ONLY — fixture/loop evidence, NOT a result; the pre-registered 11.5 metric
is not computed"`. That flag is **correct about the trainer** and must not be "fixed" — the trainer
fits on `train`/`val` and never opens the test split, deliberately, so the held-out split cannot be
contaminated by the search over 10 runs, 2 arms and 5 seeds.

The tool **refuses (exit 2)** rather than guessing when: the head or cache is missing; the split is
empty; the cache `config_hash` is not `78f1e3700da15aa1` (a different experiment); or the head cannot
be built against the frozen `(B, 2318)` input contract.

Definition (per `docs/PHASE14_CROMA_NORMALISATION_CHANGE.md` §4): *fusion-head accuracy and macro-F1
over the 19-class label space on the held-out split.*

#### Measured numbers

All from `artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json`:

| Metric | Value | Exact key path |
|---|---|---|
| **accuracy** | **0.931** | `accuracy` |
| **macro F1** | **0.434161** | `macro_f1` |
| loss | 0.254592 | `loss` |
| split | `test` | `split` |
| n scored | 4,000 | `n_scored` |
| num classes | 19 | `num_classes` |
| macro-F1 denominator | all 19 classes (absent classes contribute 0.0) | `macro_f1_denominator` |
| classes present | `[0,2,3,4,5,6,7,8,9,10,12,13,17,18]` | `classes_present` |
| classes absent | `[1,11,14,15,16]` | `classes_absent` |
| deciding statistic | **false** | `is_deciding_statistic` |
| cache arm | A | `cache_arm` |
| cache path | `artifacts/optical_sar/fusion_features/test.npz` | `cache_path` |
| cache config hash | `78f1e3700da15aa1` | `cache_config_hash` |

**Optical-SAR accuracy 0.931 must ALWAYS travel with macro-F1 0.434161.** The two are recorded side by
side in the artifact precisely so neither can be quoted alone.

Per-class F1 (`_per_class_f1`, 19 terms, indexed 0–18):

```
[1.000, 0.000, 0.428, 0.571, 0.887, 0.000, 0.000, 0.800, 0.929, 0.940,
 0.182, 0.000, 0.200, 0.438, 0.000, 0.000, 0.000, 0.877, 0.998]
```

**Two different averages come from the same per-class vector, and this is a trap:**

| Averaged over | Value |
|---|---|
| **all 19 slots** — the pre-registered definition | **0.434161** |
| the 14 present classes only | **0.589218** |

A reader who computes the second and compares it to the recorded scalar will wrongly conclude the
recorded figure is wrong. It is the 19-slot mean, and the trainer's `_macro_f1` divides by
`num_classes` by construction. Both are reproducible:
`sum(per_class)/19 = 0.43416082670034284` and
`sum(per_class[c] for c in present)/14 = 0.5892182648076082` (`docs/PHASE12_115_METRIC_COMPUTED.md`
§3.5).

**Five absent classes contribute `0.0` by definition — but two present classes also score `0.0`.**
Classes **1, 11, 14, 15, 16** are absent (their `0.0` describes no prediction); classes **5** and
**6** are **present** and score `0.0`, i.e. a **real total miss**. So the honest reading is *both*: part
of the low macro-F1 is populational, **and** there is genuine per-class failure. High accuracy with a
wide per-class spread is the signature of prediction dominated by frequent classes.

Measured test-set label distribution (`docs/PHASE12_115_METRIC_COMPUTED.md` §3.4):

```
[13, 0, 63, 39, 846, 4, 1, 6, 453, 246, 4, 0, 9, 18, 0, 0, 0, 34, 2264]
```

14 of 19 classes present; five have zero samples; top-to-bottom ratio **2,264 : 1** (class 18 vs class
6). The majority class holds 2,264 / 4,000 = **0.566**, and the head scores **0.931** — so the accuracy
is not a constant predictor. Per-class recall on the held-out split (selected rows): class 18 n=2264
recall **0.997**; class 9 n=246 recall **0.951**; class 17 n=34 recall **0.941**; class 8 n=453
**0.905**; class 4 n=846 **0.809**; class 3 n=39 **0.410**; class 12 n=9 **0.111**; classes 5 and 6
recall **0.000**.

**Four independent verification checks** (`docs/PHASE12_115_METRIC_COMPUTED.md` §3): (1) calling
`evaluate_fusion_head` directly returns `n=4000, accuracy=0.931, macro_f1=0.43416082670034284`;
(2) rebuilding the input tensor by hand — `concat([optical_gap, sar_gap, joint_gap])` then
`concat([…, optical_mask, sar_mask])`, widths `(4000,768)×3 + (4000,12) + (4000,2) = 2318` — gives
0.931 again (this checks feature *ordering*); (3) `sklearn.metrics` `accuracy_score = 0.931`,
`f1_score(average="macro", labels=range(19), zero_division=0) = 0.43416082670034284`; (4) the
per-class breakdown above.

#### Acceptance status

**OPEN.** `is_deciding_statistic: false`. The artifact's own `advisory` field states: *"This tool
reports ONE head's held-out accuracy and macro-F1. It selects no head, ranks nothing and compares no
arms. Whether this constitutes a Phase 12 pass is the owner's ruling."* Ruling 5 attaches **no
numerical threshold**; ruling 4 requires it be described as a **single-label subset evaluation, never
as a multi-label BigEarthNet/reBEN result**.

#### ⚠️ The caveat that governs how these numbers may be used

The cache metadata records `label_policy = require_single_label`, `n_skipped_by_policy = 0`. reBEN
v2.0 is a **multi-label** corpus while the frozen head is a **single-label 19-class softmax** trained
with `cross_entropy`. The extraction therefore used policy (b) — restrict to single-label patches —
which preserves the frozen architecture and the 19-class space exactly but **changes the evaluation
population**.

So this metric **may not** be presented as: a multi-label BigEarthNet/reBEN result; comparable to
published BigEarthNet numbers (almost all multi-label); or a statement about all 19 classes (**5 have
no test samples here**). It **may** be presented as: *the pre-registered 11.5 metric, as computed under
the single-label extraction policy the frozen architecture requires, on the held-out split.*

#### Limitations

- **Accuracy is carried by common classes** (0.931 vs macro-F1 0.434161; two present classes at 0.0).
- **The live service returns a bare class index** (`class_18`), not a human-readable label
  (`_label_for` falls back to `f"class_{index}"` when `class_labels` is empty).
- **Single-label subset** — not comparable to multi-label BigEarthNet numbers.
- **Ruling OPEN**; no threshold attached.
- **The confidence path floors to 0.0 when there is no prediction or no trained head** — a signal
  *gap*, not a weak signal. Measured with no checkpoint: `raw = 0.0`,
  `components = {optical_confidence: 0.333, sar_confidence: 1.0, cross_modal_agreement: 0.0,
  has_croma: 0.0, trained_head: 0.0, no_prediction: 1.0}`, `degradation_reason: "CROMA is not loaded"`.
- **Two assets must be the right two.** `validate_request` rejects 0, 1, 3, and also **two of the same
  kind** — a plausible mistake that, if accepted, would place a second optical image into the 2-channel
  SAR slot, zero-fill, run CROMA, and answer about one modality while claiming to fuse two.

---

### 3.4 `grounding` — RemoteCLIP grounding head

**Status:** `IMPLEMENTED`, `MEASURED` — **two protocols** and **two decode variants**.

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `grounding_head` | `models/manifest.json` |
| Path | `artifacts/grounding/remoteclip_grounding_v001/head.pt` | manifest |
| Bytes | 12,639,041 | manifest |
| sha256 | `93432f7034be91a8ffd9c1a84e3eeec00bed7832c043fe7f83d2be230284c6bb` | manifest, `checksums.sha256` |
| HF path | `grounding/head.pt` | manifest |
| Architecture | RemoteCLIP ViT-B/32 grounding head (feature_dim 2048, hidden 512) | manifest |
| Backbone | `chendelong/RemoteCLIP` (`RemoteCLIP-ViT-B-32.pt`, rev `bf1d8a3ccf2d`) | manifest |
| Config hash | `78f1e3700da15aa1` | manifest |
| **Parameters** | **1,052,677** | `docs/PHASE8_GROUNDING_HEAD_DECISION.md` |

#### Architecture (`specialists/grounding/head.py`)

A trainable head over the **frozen** RemoteCLIP ViT-B/32 encoder. Per-cell feature is
`concat([patch, text, patch·text, global_pool]) = 4 × 512 = 2048` (enforced at config load — §3.0.2).

```
FROZEN INPUTS (measured, docs/PHASE7_GROUNDING_CONTRACT.md):
    patch tokens : (B, 49, 512) at 224px   -- 7x7 grid, projected dim 512
    text emb     : (B, 512)
```

The head is `proj: Linear(2048, 512)`, `norm: LayerNorm(512)`, `drop: Dropout(0.10)`,
`out: Linear(512, 5)` — five outputs `[tx, ty, tw, th, objectness]`. Forward: `proj → GELU → LayerNorm
→ Dropout → out`.

**Small-std init, and objectness bias low** (`_init_weights`): `proj.weight ~ N(0, 0.02)`,
`proj.bias = 0`, `out.weight ~ N(0, 0.01)`, `out.bias = 0` except `out.bias[4] = -2.0`. With one
positive cell in 49 the task starts 1:48 imbalanced; a bias of −2.0 starts the objectness prior near
0.12, closer to the truth than 0.5, and stops the first epochs being spent un-learning a saturated
sigmoid.

**Box parameterisation — cell-relative (YOLO-style).** Cell `(r, c)` of a `grid_h × grid_w` map
predicts `[tx, ty, tw, th, obj]`:

```
cx = (c + sigmoid(tx)) / grid_w
cy = (r + sigmoid(ty)) / grid_h
w  = sigmoid(tw)
h  = sigmoid(th)
box = clip((cx - w/2, cy - h/2, cx + w/2, cy + h/2), 0, 1)
```

Cell-relative rather than absolute because 7×7 is coarse: an absolute regressor must learn 49 separate
mappings onto the same global coordinates; a cell-relative one only learns a local offset. After
clipping, `enforce_order` guarantees `x2 ≥ x1` and `y2 ≥ y1` — without it a decoded box can be
inverted, which makes IoU zero and silently kills the gradient.

**Positive assignment.** Exactly one cell per target: the one containing the ground-truth box
**centre** (standard single-stage-detector convention, YOLO/FCOS). Ties go to the earlier cell so the
assignment is deterministic. This makes the 1-of-49 confidence imbalance explicit and therefore
correctable.

**NMS** is pure torch (no torchvision dependency), so the deployed Space needs no extra package and
the behaviour is identical on CPU and GPU.

**Loss** (`grounding_loss`): `box = L1(pos_boxes, targets)` (weight 0.5),
`giou = 1 - GIoU(pos_boxes, targets)` (weight 0.3), and `conf = BCE_with_logits(obj_logits,
conf_targets, weight=weights)` (weight 0.2) where the single positive cell is up-weighted by
`positive_confidence_weight = 20.0`. Box and GIoU are computed on the **decoded** coordinates of the
single positive cell per sample, so the loss is computed on exactly the coordinates the metric
measures. GIoU is used alongside L1 because L1 alone is scale-blind. `LossBreakdown` keeps the
components separate so a collapse in one is visible in the logs.

#### Hyperparameters — every value

From `configs/base.yaml` (`grounding:`, `grounding_head:`, `grounding_training:`):

| Key | Value |
|---|---|
| `grounding.checkpoint_repo` | `chendelong/RemoteCLIP` |
| `grounding.checkpoint_file` | `RemoteCLIP-ViT-B-32.pt` |
| `grounding.checkpoint_revision` | `bf1d8a3ccf2d` |
| `grounding.model_name` | `ViT-B-32` |
| `grounding.image_size` | **224** |
| `grounding.resolution_frozen` | true |
| `grounding.nms_iou` | 0.50 |
| `grounding.max_candidates` | 20 |
| `grounding.confidence_threshold` | 0.40 |
| `grounding.benchmark_box_scale` | 100.0 |
| `grounding.coordinate_system` | `normalized_0_1` |
| `grounding.encoder_projected_dim` | 512 |
| `grounding_head.hidden_dim` | 512 |
| `grounding_head.dropout` | 0.10 |
| `grounding_head.feature_dim` | 2048 |
| `grounding_head.positive_confidence_weight` | **20.0** |
| `grounding_head.decode` | `cell_relative` |
| `grounding_training.learning_rate` | 0.0001 |
| `grounding_training.batch_size` | 16 |
| `grounding_training.epochs` | 20 |
| `grounding_training.weight_decay` | 0.0001 |
| `grounding_training.warmup_ratio` | 0.05 |
| `grounding_training.grad_clip` | 1.0 |
| `grounding_training.val_fraction` | 0.10 |
| `grounding_training.num_workers` | 2 |
| `grounding_training.save_every_steps` | 500 |
| `grounding_training.box_loss_weight` | 0.5 |
| `grounding_training.giou_loss_weight` | 0.3 |
| `grounding_training.confidence_loss_weight` | 0.2 |

**Why `positive_confidence_weight = 20.0`.** Objectness BCE sees ~1 positive cell out of 49.
Unweighted, the optimum is "no object" everywhere; the weight is what stops that collapse.

#### Training data

VRSBench. The head trains on the train split and is evaluated on the VRSBench **eval** split, n =
**16,159** records, scored once, no sampling. VRSBench stores boxes normalised to **0–100**; this
project stores boxes normalised to **0–1**, via the declared `benchmark_box_scale: 100.0` so the
conversion cannot be applied twice or forgotten.

**Validation trajectory** (20 epochs, CPU, ~37 s/epoch) — `docs/PHASE8_GROUNDING_HEAD_DECISION.md`:

```
epoch   1/20  loss 0.4284  box 0.0641  giou 0.7948  conf 0.7897  val_iou 0.0436
epoch   5/20  loss 0.3864  box 0.0489  giou 0.7581  conf 0.6725  val_iou 0.0673
epoch  10/20  loss 0.3653  box 0.0446  giou 0.7313  conf 0.6179  val_iou 0.0831
epoch  15/20  loss 0.3488  box 0.0414  giou 0.7062  conf 0.5811  val_iou 0.0926
epoch  20/20  loss 0.3419  box 0.0401  giou 0.6937  conf 0.5685  val_iou 0.0943
```

All four loss components move monotonically in the right direction. `conf` falls from an
over-confident 0.79 as the positive-cell weighting corrects the objectness prior; `box` and `giou` fall
together, which is the healthy pattern (a falling `box` with a rising `giou` would mean the box is
drifting in size). **Validation IoU (0.0943 at epoch 20) is much lower than eval IoU (0.2566).** These
are not comparable and the gap is expected: validation is a held-out slice of the **train** split,
which has noisier ground truth (29.5 % of its boxes are out-of-range and filtered, and the surviving
ones come from a different annotation pass). This is a property of VRSBench, not a bug.

#### Evaluation protocol — two protocols × two decode variants

`artifacts/grounding/remoteclip_grounding_v001/eval_result_canonical.json` (top_k = 20, the config
default) and `…/eval_result_matched6.json` (top_k = 6, decode-matched to the baseline). Both record
`n_eval_records: 16159`, `resolution: 224`, `grid: 7`, `limited_run: false`,
`frozen_config_evaluation: true`, device `cpu`, torch `2.14.0+cpu`, and
`config_hash: 78f1e3700da15aa1`, `config_drift: false`.

**The `head_decode` block is recorded in each artifact** — `config_default_top_k: 20`, `nms_iou: 0.5`,
`score_threshold: 0.4`, `top_k: 20` (canonical) or `top_k: 6` (matched6). Without it the artifact could
not say which setting produced its number, and a re-run at the config default would silently yield a
different figure.

#### Measured numbers

**The trained head, under the two protocols:**

| Protocol | mean best IoU | recall@0.5 | Artifact key path |
|---|---|---|---|
| canonical (head threshold decode, top_k = 20) | **0.2838** | **0.2198** | `results.head_threshold.mean_best_iou` / `results.head_threshold.recall.0.50` |
| matched6 (head threshold decode, top_k = 6) | **0.2566** | **0.1938** | `results.head_threshold.mean_best_iou` / `results.head_threshold.recall.0.50` |

**Two decode variants, reported for completeness** (a reviewer must see the whole grid, not one cell):

| Variant | mean best IoU | recall@0.10 | recall@0.25 | recall@0.50 |
|---|---|---|---|---|
| head argmax decode (canonical) | **0.1215** | 0.3183 | 0.2088 | 0.0795 |
| zero-shot matched (no trained head, canonical) | **0.0972** | 0.3298 | 0.1188 | 0.0234 |

**Full canonical grid** (`eval_result_canonical.json` → `results`):

| Strategy | `mean_best_iou` | `recall.0.10` | `recall.0.25` | `recall.0.50` | `latency_ms_per_image` | `seconds` |
|---|---|---|---|---|---|---|
| `head_argmax` (strategy `argmax`) | 0.1215 | 0.3183 | 0.2088 | 0.0795 | 0.655 | 10.6 |
| `head_threshold` (strategy `threshold`, top_k 20) | 0.2838 | 0.6882 | 0.5047 | 0.2198 | 2.205 | 35.6 |
| `zero_shot_matched` | 0.0972 | 0.3298 | 0.1188 | 0.0234 | — | 17.9 |

**Full matched6 grid** (`eval_result_matched6.json` → `results`): `head_argmax` identical (0.1215);
`head_threshold` (top_k 6) 0.2566 / 0.6315 / 0.4545 / 0.1938 at 2.158 ms; `zero_shot_matched`
identical (0.0972). Each artifact also carries `phase7_reference: {mean_best_iou: 0.0972,
recall_at_0.50: 0.0234, source: "docs/PHASE7_RESOLUTION_DECISION.md"}`.

**Decode-matched delta (head threshold top_k = 6 vs zero-shot): +0.1594 IoU, +0.1704 Recall@0.50.**
The pre-registered bar was `MIN_IMPROVEMENT_IOU = 0.02`; the measured margin is **8× the bar**.

**`head_argmax` vs zero-shot is NOT apples-to-apples.** 1 box against ~6 boxes flatters the head,
because mean best IoU takes the max over predictions. It is reported because it is the number directly
comparable to the Phase 7 resolution experiment's zero-shot argmax, not because it decides anything.

**A defect this phase had to fix first** (`docs/PHASE8_GROUNDING_HEAD_DECISION.md`). The evaluation
script originally built its own single-box baseline with `argmax_candidate`, while Phase 7 measured the
baseline through `ground_phrase` (threshold box + up to 5 local maxima). Same 16,159 records, same
metric, same cached features — **different decode**: Phase 7 via `ground_phrase` = 0.0972, eval via
`argmax_candidate` = 0.0092, a 10× gap. The eval printed `head beats zero-shot: True (+0.1123)` when
the matched comparison was +0.0243. Both cleared the bar, but only the second is a claim about the head
rather than about the decode. **Fix:** the decode was extracted into one function both paths call —
`specialists/grounding/inference.py::decode_candidates_from_features` — so there is exactly one place
that turns similarity into boxes. Verification that the fix is real, not cosmetic: re-running the eval
reproduced the Phase 7 number exactly (`|difference| = 0.0000`).

**A second defect: candidate count changes the number.** Mean best IoU is a max over predictions, so
emitting more boxes raises it mechanically. The head's threshold decode defaulted to `max_candidates:
20` while the baseline emits 5.99 — an uncontrolled asymmetry in the head's favour. Fix:
`--head-top-k` and `--head-score-threshold` overrides, plus the recorded `head_decode` block. Measured
cost of the cap: top_k 20 → 0.2838 (+0.1866 vs zero-shot); top_k 6 → 0.2566 (+0.1594). Capping to the
baseline's own budget costs 0.027 IoU — **the win survives.**

#### Acceptance status

`MEASURED`. Phase 8 verdict: **the trained head beats the zero-shot baseline**; Phase 8 is done. The
head is **not yet wired into the specialist** as of the Phase 8 decision record
(`specialists/grounding/inference.py` still exposes only the zero-shot path), and that integration step
must not change the zero-shot module's behaviour because the resolution experiment depends on it.

#### Limitations

- **Absolute IoU is low.** 0.2838 / 0.2566 is not "solved".
- **Protocol-sensitive.** 0.2838 (canonical) vs 0.1215 (argmax) — absolute values depend on decode.
- **Localisation floor.** At 224 each token covers 1/7 (~0.143) of the image width; at 448 it is 1/14.
- **`head_threshold` at top_k = 20 was not tuned.** 20 is the config default, not a
  validation-selected optimum. Selecting it on eval would be benchmark tuning.
- **Nothing about the hidden distribution.** VRSBench is overhead optical; the hidden set is
  Cartosat-2S + RISAT.
- **Not a system-level result.** "Beats the zero-shot baseline on VRSBench eval" ≠ "performs well on
  the hidden ISRO/SAC set".

---

### 3.5 `router` — intent adapter over frozen MiniLM

**Status:** `IMPLEMENTED`, `MEASURED`, **TEST NOT RUN**.

A **~50,822-parameter adapter** over the frozen MiniLM encoder. Because the encoder is frozen,
embeddings are cached and the adapter trains on cached vectors — no GPU required (measured: 20 epochs /
4,096 vectors in 0.28 s on CPU).

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `router_adapter` | `models/manifest.json` |
| Path | `artifacts/router/router_adapter_v001/adapter.pt` | manifest |
| Bytes | 211,961 | manifest |
| sha256 | `8527c3ed28a293e13293d48601d48e3ceafa137b9acabddaf5de31a58a509b5c` | manifest, `checksums.sha256` |
| HF path | `router/adapter.pt` | manifest |
| Architecture | task/modality adapter over frozen MiniLM embeddings (~50,822 params) | manifest |
| Backbone | `sentence-transformers/all-MiniLM-L6-v2` (rev `1110a243fdf4`) | manifest |
| Config hash | `78f1e3700da15aa1` | manifest |
| Parameters | `UNKNOWN — not established from the available evidence` (manifest records `null`) | manifest |

> **Parameter-count discrepancy — flagged, not smoothed over.** `specialists/router/adapter.py`'s
> docstring and `models/manifest.json` both state **~50,822** parameters. The Phase 4 completion
> report (`docs/PHASE4_ROUTER_REPORT.md`) states **51,725** adapter parameters on a frozen
> 22,713,216-param encoder and an artifact size of 223.8 KB, whereas the released artifact is
> 211,961 bytes. These two figures were produced at different times and have not been reconciled.
> The release documents the **manifest** figure (50,822) as the shipped number; which figure is
> authoritative is
> `UNKNOWN — not established from the available evidence`.

#### Architecture (`router/adapter.py`)

The only trainable part of the router. It sits on top of the frozen MiniLM embedding and emits **five
heads**:

```
embedding (384)
    |
LayerNorm
    |
Linear(384 -> hidden_dim)      default hidden_dim = 128
    |
GELU
    |
Dropout
    |
    +--> task_head            Linear(hidden, 6)
    +--> modality_head        Linear(hidden, 4)
    +--> temporal_head        Linear(hidden, 1)   logit
    +--> spatial_head         Linear(hidden, 1)   logit
    +--> language_head        Linear(hidden, 1)   logit
```

`_init_weights` uses small-std init (`std = 0.02`) with zero bias on every head, keeping the initial
sigmoid near 0.5 — without it the binary heads can start saturated and BCE gradients vanish before the
task head has learned anything useful. `forward` asserts the input is 2-D and that
`embeddings.shape[1] == input_dim`, and requires embeddings already detached from the frozen encoder
(the adapter does not back-propagate into MiniLM).

#### Hyperparameters — every value

From `configs/base.yaml` (`router:`):

| Key | Value |
|---|---|
| `model` | `sentence-transformers/all-MiniLM-L6-v2` |
| `revision` | `1110a243fdf4` |
| `max_length` | 128 (asserted ≤ 256) |
| `embedding_dim` | 384 |
| `device` | `auto` |
| `hidden_dim` | 128 |
| `dropout` | 0.10 |
| `confidence_threshold` | 0.70 |
| `num_tasks` | 6 |
| `tasks` | `[vqa, caption, grounding, change, optical_sar, unsupported]` |
| `training.epochs` | 60 |
| `training.batch_size` | 64 |
| `training.learning_rate` | 0.001 |
| `training.weight_decay` | 0.01 |
| `training.task_loss_weight` | 1.0 |
| `training.modality_loss_weight` | 0.3 |
| `training.binary_loss_weight` | 0.5 |
| `training.val_ratio` | 0.15 |
| `training.hard_negatives_to_test` | true |

**Splits are by GROUP (template / hard-negative family), never by example.** Template-generated
queries are near-duplicates; splitting by example would put `"Show me the water body."` in train and
`"Show me the road."` in val — one token apart — and report a fake accuracy. Hard-negative families
are placed in the **test** split so their accuracy measures generalisation, not memorisation.

#### Training data

A **synthetic** corpus: 576 examples, 54 groups, hand-written and templated
(`threshold_sweep_val.json`: `corpus_total: 576`, `corpus_groups: 54`, `corpus_limited: true`). Split
sizes: `train 410`, `val 86`, `test 80`. The plan's minima are `plan_min_val_queries: 500` and
`plan_min_hard_negatives: 100`; the corpus is well below both, and the artifact says so.

#### Evaluation protocol and measured numbers

`artifacts/router/threshold_sweep_val.json`. The sweep iterates thresholds 0.50 → 0.99 and records
coverage, covered-task accuracy, fallback rate and `n_covered` per threshold. `select_by:
"covered_accuracy"`; `selected.threshold: 0.76` (coverage 0.790698, covered-task accuracy 1.0);
`shipped_threshold: 0.70` (coverage 0.848837, covered-task accuracy 0.972603).

| Field | Value | Exact key path |
|---|---|---|
| **overall ungated task accuracy** | **0.965116** | `overall_ungated_accuracy` |
| split | `val` | `split` |
| n val | **86** | `n_val` |
| n val examples scored | 86 | `n_val_examples_scored` |
| n test examples scored | **0** | `n_test_examples_scored` |
| `test_split_touched` | **false** | `test_split_touched` |
| `corpus_limited` | true | `corpus_limited` |
| `hard_negatives_in_val` | **0** | `hard_negatives_in_val` |
| `val_min_support` | 8 | `val_min_support` |
| `val_task_counts` | `{caption: 8, change: 20, grounding: 14, optical_sar: 10, unsupported: 19, vqa: 15}` | `val_task_counts` |
| `split_sizes` | `{train: 410, test: 80, val: 86}` | `split_sizes` |
| `delta_vs_shipped.coverage` | −0.0581 | `delta_vs_shipped.coverage` |
| `delta_vs_shipped.covered_task_accuracy` | 0.0274 | `delta_vs_shipped.covered_task_accuracy` |
| adapter encoder params | 22,713,216 | `adapter_encoder.parameters` |
| `adapter_config_hash` | `615478910dc266bf` | `adapter_config_hash` |
| `config_hash` | `78f1e3700da15aa1` | `config_hash` |

**This number is (a) validation-only, (b) ungated, and (c) small (n = 86).** The artifact's own note is
blunt: *"corpus-limited: val n=86 vs plan >=500. This is NOT a calibration — the corpus is synthetic
and too small (min per-class support 8, caption) and val carries 0 hard negatives (hn_* families are
held out to TEST by design). Selecting a threshold here yields a justified default, not a calibrated
value. The corpus was NOT padded with generated queries. Backlog P1-9's 'n=80' is the TEST split; the
sweep target is val n=86. **The test split was NOT touched.**"*

**The router TEST split was NOT RUN.** Do not read 0.965116 as a test result. The sweep scored 86
validation examples and zero test examples, and `test_split_touched` is `false`.

> **Historical note, recorded so the two are not conflated.** The Phase 4 completion report
> (`docs/PHASE4_ROUTER_REPORT.md`, verified 2026-09-16, Gate 2 PASS) records an **earlier** gate-2
> evaluation that included a test column — test task accuracy **0.975**, macro F1 0.976, per-class test
> recall (vqa 1.000/n=15, caption 0.846/n=13, grounding 1.000/n=17, change 1.000/n=13, optical_sar
> 1.000/n=12, unsupported 1.000/n=10) and hard-negative accuracy **0.800**. That evaluation predates
> the shipped threshold sweep, and its own standing caveats are that the corpus is synthetic and that
> *"the 0.975 headline is partly earned on templates the split kept in training. Treat the router as
> working, not as benchmarked."* The release's position for the shipped artifact is **TEST NOT RUN**;
> the Phase 4 numbers are retained as a historical record, not promoted to a release benchmark.

#### Acceptance status

`MEASURED`; the headline is validation, ungated, n = 86. No test result is claimed.

#### Limitations

- **Corpus-limited** (n = 86 val; plan minimum 500; hard negatives in val = 0 by design).
- **Not a calibration** — the threshold is a justified default, not a calibrated value.
- **Synthetic corpus** — 576 hand-written/templated examples.
- **`caption` has the smallest support** (8 in val), the tightest per-class floor.
- **Known misroutes** are catalogued in [`LIMITATIONS.md`](LIMITATIONS.md) §2 (e.g. *"What is the new
  runway?"* reads `change`, not `vqa`; *"How much built-up area was added?"* under-triggers to `vqa`).
- **Threshold 0.70 is uncalibrated** — it gates the lexical fallback.

---

### 3.6 `vlm` — SmolVLM LoRA adapter (USABLE_VERIFIED, ACCEPTANCE-REJECTED)

**Status:** `IMPLEMENTED`, `MEASURED`, **ACCEPTANCE-REJECTED**.

> This is §3.6 because [`TRAINING.md`](TRAINING.md) §7 cross-references it. The section number is kept
> stable so that link resolves.

A PEFT LoRA adapter on frozen `HuggingFaceTB/SmolVLM-500M-Instruct`. **The artifact's status is
`CLOSED` with headline `ACCEPTANCE-REJECTED`.** This is not a contradiction — it is the project's
central truthfulness distinction: **`USABLE_VERIFIED`** (the adapter demonstrably works; the metrics
are real and reproducible) and **`ACCEPTANCE-REJECTED`** (it is *not accepted* for production
promotion). **USABLE ≠ ACCEPTED.** The deployed caption/VQA path therefore uses the **unadapted**
SmolVLM.

#### Identity

| Field | Value | Source |
|---|---|---|
| Manifest `id` | `vlm_lora_adapter` | `models/manifest.json` |
| Path | `.scratch/phase6_real_adapter/phase6_adapter/adapter_model.safetensors` | manifest |
| Bytes | 34,798,048 | manifest |
| sha256 | `07c76a75fa04624880ed7730590f5fdd7b145a8232e3c0af411c3c545a5adf5e` | manifest, `checksums.sha256` |
| HF path | `vlm/adapter_model.safetensors` | manifest |
| Architecture | PEFT LoRA (r=16, alpha=32, dropout=0.05) on text_model projections | manifest |
| Base model | `HuggingFaceTB/SmolVLM-500M-Instruct` (rev `a7da5b986cb5`) | manifest, closure |
| `kind` | `bigearthnet_smolvlm_lora` | closure → `production_adapter.kind` |
| **tree sha256** | `5c6b86317d1e65962702dc9e377009b3df41cc13de1b15bceccb70ad977775e7` | closure → `adapter_tree_sha256` |
| **file sha256** | `07c76a75fa04624880ed7730590f5fdd7b145a8232e3c0af411c3c545a5adf5e` | closure → `weights_file_sha256` |
| Trainable params | **8,683,520** (1.6823 % of 516,165,824) | closure → `trainable_params` |
| Config hash | `78f1e3700da15aa1` | manifest |
| `retrained_for_closure` | **false** | closure |
| `modified_for_closure` | **false** | closure |

#### Architecture and target modules

LoRA, `peft_type=LORA`, `r=16`, `alpha=32`, `dropout=0.05`, targeting
`model.text_model.*.{q,k,v,o,gate,up,down}_proj` — `lora_target_module_count: 224`.

**The vision-tower hazard did not occur.** `trainable_subtrees` is exactly
`{"model.text_model": 8683520}`; the vision model (86,433,024) and connector (11,796,480) appear in
`frozen_params`, which are:

| Frozen subtree | Parameters |
|---|---|
| `model.connector` | 11,796,480 |
| `model.text_model` | 361,944,000 |
| `model.vision_model` | 86,433,024 |
| `other` | 47,308,800 |

The `trainable_params` figure was measured two independent ways (closure → `trainable_params.how`):
summing `numel` over every tensor in `adapter_model.safetensors` read from the safetensors header
(no model load), and loading through the production path and reading `sum(p.numel())` then subtracting
the manifest frozen-params sum. Both give 8,683,520; `manifest_matches_measurement: true`.

#### Hyperparameters — every value

From `configs/base.yaml` (`training:`) and the closure record:

| Key | Value |
|---|---|
| `training.precision` | **fp16** (finding C-6: T4 is SM 7.5 → fp16, NOT bf16) |
| `training.vlm_batch_size` | 2 |
| `training.vlm_gradient_accumulation` | 8 |
| `training.vlm_learning_rate` | 0.0002 |
| `training.vlm_epochs` | 1 |
| `training.lora_rank` | 16 |
| `training.lora_alpha` | 32 |
| `training.lora_dropout` | 0.05 |
| `training.weight_decay` | 0.01 |
| `training.warmup_ratio` | 0.05 |
| `training.gradient_checkpointing` | true |
| `training.save_every_steps` | 500 |
| PEFT version | **0.19.1** |
| Seed | 42 |
| `precision_recorded` | fp16 |

#### Training data

BigEarthNet instruction pairs (the same 28,000-patch local subset used for the fusion head), on an
external GPU (Kaggle T4-class). This is the only artifact that requires a GPU to train.

#### Evaluation protocol

`artifacts/vlm/phase6_closure.json` (generated by `scripts/phase6_close.py`, which reads the evidence
rather than restating it). The evaluation is on a **frozen 1,000-question subset** of the test split.
The subset's identity was proven **without a model** (Gate A″): `available_per_split {val: 6750, test:
7772}`, subset `n = 1000`, and the test per-class counts an **exact** match to the recovery subset
(19 classes, sum 1000), `config_hash 78f1e3700da15aa1` identical to Run 1's.

Gate D reproduced Run 1's adapted-test control **exactly** — loading the local adapter and evaluating
it returned `exact_match 0.963`, `n 1000`, confusion `{tp:500, fp:19, tn:463, fn:18}`,
`f1 0.9643201542912246` — identical to Run 1's recorded values, which proves the local artifact *is*
Run 1's adapter and that CPU/fp32 reproduces the Kaggle T4 endpoint.

#### Measured numbers — the "usable" side

`artifacts/vlm/phase6_closure.json` → `why_usable_verified.adapted_test`:

| Metric | Value | Exact key path |
|---|---|---|
| **exact_match** | **0.963** | `why_usable_verified.adapted_test.exact_match` |
| **F1** | **0.96432** | `why_usable_verified.adapted_test.f1` |
| n | 1000 | `…adapted_test.n` |
| precision | 0.963391 | `…adapted_test.precision` |
| recall | 0.965251 | `…adapted_test.recall` |
| tp / fp / tn / fn | 500 / 19 / 463 / 18 | `…adapted_test.confusion` |
| aggregate test delta | **+49.50 pp** (46.80 → 96.30 pp) | `why_usable_verified.aggregate_test_delta_pp` |

Per-class accuracy on the adapted test subset ranges from 0.8 (Agro-forestry areas, n=20) to 1.0
(several classes); `Mixed forest` 0.878788 (n=33), `Permanent crops` 0.862069 (n=29), `Broad-leaved
forest` 0.898551 (n=69), `Pastures` 0.916667 (n=132), `Marine waters` 1.0 (n=329).

#### Measured numbers — the "rejected" side

`artifacts/vlm/phase6_closure.json` → `why_acceptance_rejected`. Decision split: **test**. Rule
version: **v002**.

| | |
|---|---|
| **V1** | **passes** — test `46.80 → 96.30 pp`, delta **+49.50 pp** (required ≥ +5.00) |
| **V2** | **fails** — **Mixed forest**: `n=33`, `100.00 → 87.8788 pp`, `drop 12.1212 pp`, `lost_questions 4`, `z 2.1335` |

It fails **both** halves of v002 (`lost ≥ 4` **and** `z ≥ 1.96`). Per item V, a complete run that fails
V2 is `REJECTED`.

Thresholds used (`why_acceptance_rejected.thresholds_used`): `accept_min_delta_pp: 5.0`,
`accept_min_delta_pp_ceiling: 2.0`, `ceiling_baseline_pp: 95.0`, `max_class_drop_pp: 1.0` (legacy
v001), `min_class_questions: 20`, `test_val_disagreement_pp: 10.0`; `v2_criterion`: `class_drop_z:
1.96`, `min_class_drop_questions: 4`, `min_class_questions: 20`, with
`se_formula: "sqrt((p_b*(1-p_b) + p_a*(1-p_a)) / n)"` and `z_formula: "(baseline - adapted) / se"`.

**The rejection is narrow.** `n_classes_failed: 1`, `n_classes_held: 5`, `n_classes_improved: 11`,
`n_classes_total: 19`. The next-worst class (Inland wetlands, −6.6667 pp, n=30) lost only 2 questions
and therefore sits below V2's materiality floor.

**It is not a split artefact.** The same class also degraded on **val** in Run 1 (drop 6.4516 pp,
n=31) — the very value that motivated v001's flag. The adapter hurts Mixed forest on **both** splits,
so this is a property of the adapter, not an accident of one subset. Mixed forest also sits at a
100.00 pp baseline on test, so any loss is a drop from the ceiling.

**Residual risk, reported not resolved** (`residual_risk`): the verdict rests on **4 questions in one
class of 33** — the unfloored minimum-size exposure recorded at `PHASE6_AUDIT_AND_CONTRACT.md` §8.6.
With no `n ≥ N` floor in V2, a 33-question class can flip the verdict of a run whose aggregate endpoint
improved by 49.5 pp.

**BERTScore is unavailable**, not zero: `thresholds_used.bertscore.available: false`, reason
`roberta-large` is not in the local HuggingFace cache. BLEU/ROUGE are **excluded** because target
answers are one token and they are meaningless at that length.

#### Both rejection records are preserved

Neither is rewritten. A future reader must be able to see what each rule said, on which split, at the
time it said it (`preserved_records`):

| Record | Rule | Split | Verdict | Source |
|---|---|---|---|---|
| Run 1's own manifest | `v001` | val | **`REJECTED`** (3 classes) | Run 1's original manifest, preserved verbatim as `preserved_records.v001_val_rejected` in `phase6_closure.json` |
| Test-split adjudication | `v002` | **test** (independent) | **`REJECTED`** (1 class) | `artifacts/vlm/run1_test_recovery/test_adjudication.json` |
| Recovery manifest | `v002` | val | `ACCEPTED` — **not final acceptance** | `artifacts/vlm/run1_test_recovery/run_manifest.json` |

**v001 on val** failed three classes — Mixed forest (6.4516 pp, n=31), Transitional woodland shrub
(9.375 pp, n=32), Agro-forestry areas (4.6512 pp, n=43) — with `val_delta_pp 42.0` and
`test_delta_pp null`. **The val-split `ACCEPTED` under v002 is recorded for completeness only and is
NOT final acceptance** — it decides on the *same val subset that motivated v002*, which
`docs/PHASE6_RUN1_REJECTION_DIAGNOSIS.md` §7.4 condition 3 forbids. On data that did not motivate the
relaxation, the run does not pass.

#### Acceptance status — and what closure does NOT claim

`status: CLOSED`; `headline: "Phase 6 is closed. The Run 1 LoRA adapter is promoted to the production
VLM adapter: USABLE and VERIFIED, but ACCEPTANCE-REJECTED."` The record keeps verified and accepted
separate: *"'Verified' answers: is this artifact the one we trained, and does it work? 'Accepted'
answers: did it clear the bar predeclared before we looked? Both are true, and they are different
questions."*

`what_closure_does_not_claim`:

- It does **not** claim Run 1 was accepted — it was rejected by v001 on val and by v002 on the
  independent test split.
- It does **not** claim the Mixed forest regression is resolved.
- It does **not** claim a new adapter exists or is planned.
- It does **not** alter the v002 rule or its verdict.

**Forward rule** (`forward_rule`): any future improved adapter MUST be a **new** experiment/version. It
MUST NOT rewrite, amend, or supersede Run 1's records; it must not retrain or modify the Run 1 adapter
in place; it must not report a new adapter's metrics under Run 1's identity; it must not edit v001's or
v002's recorded verdicts to match a later outcome. Precedent: v002 itself followed this rule.

#### How it is enabled, and the two traps

The adapter is attached via an **environment setting, not a code change**:
`specialists/vqa/model.py` resolves it in the order explicit `adapter_path` argument →
**`SATQUERY_VLM_ADAPTER`** (`ADAPTER_ENV_VAR`) → none, then attaches it with
`PeftModel.from_pretrained`. A load failure raises rather than silently serving the base model: an
inference result attributed to an adapter that did not actually attach is worse than a hard failure.

Two traps a future reader will hit (`known_traps`):

1. **`adapter_sha256` names two different values.** `training/vlm/artifact.py` computes a **tree hash**
   over the `{relpath: sha256}` weight map (`5c6b8631…`), while
   `specialists/vqa/model.py::_adapter_sha256` computes the **file** sha256 of
   `adapter_model.safetensors` (`07c76a75…`). Recomputing one and comparing it to the other yields a
   false "the artifact was altered" conclusion.
2. **The promoted adapter is NOT `checkpoint-2000`.** The three weight files have three different
   digests: top-level `07c76a75…`, `checkpoint-1500` `7273588e…`, `checkpoint-2000` `bf249943…`. So
   "just use the last checkpoint" is not equivalent to this artifact.

#### Limitations

- **Acceptance-rejected** — the deployed path uses the unadapted model.
- **Residual risk** — the verdict rests on 4 questions in one class of 33.
- **`Mixed forest` regression unresolved**; it degrades on both val and test.
- **The path lives under `.scratch`** — `.gitignore` excludes `artifacts/`, `checkpoints/` and
  `*.safetensors`, so the ~105 MB local directory is not committed; a future cleanup could remove it.
  It is reconstructible from `phase6_realbundle.zip` and verifiable against the two digests above.
- **BERTScore not computed** (unavailable offline); BLEU/ROUGE excluded as meaningless at one-token
  answers.

---

## 4. Rejected, deferred and open model decisions

| Decision | Outcome | Evidence |
|---|---|---|
| Grounding image resolution 448 vs 224 | **224 chosen; 448 REJECTED** | `docs/PHASE7_RESOLUTION_DECISION.md`; §4.1 below |
| VLM adapter promotion | **REJECTED** | metrics usable, acceptance rejected (§3.6) |
| Calibration | **kept but ineffective** | ECE worsened (§5) |
| Optical-SAR ruling | **OPEN** | no decision recorded; `is_deciding_statistic: false` |
| Change-VQA ruling | **OPEN** | `verification.metric_ruling` |
| CROMA constructor (`use_croma.PretrainedCROMA`, vendored) | **ACCEPTED** as an implementation detail | `docs/PHASE14_OPTICAL_SAR_DECISIONS.md` §2 (DEV-1) |
| CROMA input normalisation | **RESOLVED** 2026-09-18: option (a) adopted; (c) gated experiment; (b) rejected | `docs/PHASE14_CROMA_NORMALISATION_CHANGE.md` (DEV-2) |
| CROMA patch count (225) | **CONSISTENT but UNVERIFIED** upstream; checked at load time | `docs/PHASE14_OPTICAL_SAR_DECISIONS.md` §4 (DEV-3) |
| `sa_mode: BAM` | **declared, NOT IMPLEMENTED** — raises rather than aliasing PAM | `specialists/change/stanet.py` |
| Change head wired into serving by default | **DEFERRED** — registry override, not config | `specialists/change/specialist.py` |
| Grounding head wired into `inference.py` | **DEFERRED** (Phase 8 integration step) | `docs/PHASE8_GROUNDING_HEAD_DECISION.md` |
| `optical.normalization` / `sar.representation` config keys | **OPEN** — read by no code | `docs/PHASE14_OPTICAL_SAR_DECISIONS.md` §6 item 6 |

### 4.1 The grounding resolution experiment — 448 REJECTED

**Rule, fixed before the result was seen** (`docs/PHASE7_RESOLUTION_DECISION.md`; recorded in the
artifact as `rule_changed_since_preregistration: false`):

```
448 WINS  if Recall@0.5 improves by >= 0.05 absolute
          OR mean best IoU improves by >= 0.05 absolute
224 WINS  otherwise
INCONCLUSIVE if fewer than 30 samples were scored
```

**Run:** full VRSBench eval split, **16,159 / 16,159 records scored at both resolutions**, Tesla T4,
`--all --device cuda --tag full`.

**Result: 224 WINS.**

```
recall@0.5 gain 448/224 : -0.0022
bestIoU    gain 448/224 : -0.0147
latency          ratio  : 1.59x
```

Neither component came close to the +0.05 margin. Both were **negative**.

| Metric | 224 | 448 | delta |
|---|---|---|---|
| token grid | 7 × 7 = 49 | 14 × 14 = 196 | 4.0× tokens |
| attention cost (n²) | 1× | 16× | — |
| with boxes | 16159/16159 | 16159/16159 | — |
| **mean best IoU** | **0.0972** | 0.0825 | **−0.0147** |
| Recall@0.10 | **0.3298** | 0.2599 | **−0.0699** |
| Recall@0.25 | **0.1187** | 0.0944 | **−0.0243** |
| Recall@0.50 | **0.0234** | 0.0212 | **−0.0022** |
| matched IoU | 0.0972 | 0.0825 | −0.0147 |
| latency mean | **20.0 ms** | 31.8 ms | **1.59×** |
| latency p90 | **20.9 ms** | 32.9 ms | 1.57× |
| peak VRAM | **592.1 MB** | 599.8 MB | +7.7 MB |
| wall time | **~8.5 min** | ~11.2 min | 1.32× |

**448 is worse on every quality metric and slower. There is no axis on which it wins.**

**Best-IoU distribution** — a whole-distribution move toward the zero-overlap bucket, not a tail
effect:

| bucket | 224 | 448 |
|---|---|---|
| 0.00–0.10 | 10,829 | 11,957 |
| 0.10–0.25 | 3,412 | 2,677 |
| 0.25–0.50 | 1,540 | 1,182 |
| 0.50–0.75 | 336 | 307 |
| 0.75–1.01 | 42 | 36 |

**Paired analysis — independent confirmation.** Both resolutions scored the **same 16,159 samples**, so
the paired test is the stronger statistic: it removes between-object variance.

```
paired samples     : 16159
mean 224           : 0.0972
mean 448           : 0.0825
mean paired diff   : -0.0147   (95% CI -0.0160 .. -0.0134)
t statistic        : -22.63
CI excludes zero   : True

448 better on      :  1371/16159 ( 8.5%)
448 worse on       :  3372/16159 (20.9%)
identical          : 11416/16159 (70.6%)
```

**The pre-registered rule and the paired test AGREE on 224.** There is no rule-versus-evidence
disagreement to escalate: both say 224, and the confidence interval excludes zero by a wide margin. The
win/loss split is also informative — 448 wins on only 8.5 % of records and loses on 20.9 %; the finer
grid is not merely neutral, it is **actively harmful** on a fifth of the corpus.

**Paired recall ladder** — the gap **narrows as the threshold rises**, the signature of a method that
cannot reach high IoU either way:

| threshold | 224 | 448 | diff | 95% CI |
|---|---|---|---|---|
| 0.10 | 0.3298 | 0.2599 | −0.0699 | excludes zero |
| 0.25 | 0.1187 | 0.0944 | −0.0243 | excludes zero |
| 0.50 | 0.0234 | 0.0212 | −0.0022 | excludes zero |

**Why 448 did not help — the honest reading.** The zero-shot method selects a patch by text similarity
and returns that patch's box. At 224 a box is 1/7 of the image; at 448 it is 1/14. A finer grid is only
better if the target is small **and** the similarity peak lands on the correct fine cell. Two things
work against that here: (1) the peak is not sharper at 448 — splitting each cell into four gives four
chances to pick a wrong sub-cell, and the similarity field on frozen features is smooth, so the argmax
moves around (losses outnumber wins 2.5 : 1); (2) **Recall@0.10 drops the most (−0.0699)** — if finer
tokens genuinely localised better, the *loosest* threshold would benefit most. It degrades most, which
means the fine grid adds positional noise rather than positional precision. **This is the zero-shot
baseline's limitation, not a property of RemoteCLIP.**

**Degeneracy notes: none.** At n=12, n=40 and n=6 the smoke runs reported `Recall@0.5 = 0.0000` at
**both** resolutions and emitted a degeneracy warning. At full scale the metric is non-zero
(0.0234 / 0.0212), so the note correctly did not fire — a genuine sub-floor artefact that the full run
resolved, which is why the sub-floor runs were never treated as evidence.

**What this establishes:** grounding runs at 224, frozen in `configs/base.yaml`
(`grounding.image_size: 224`, `grounding.resolution_frozen: true`); peak VRAM for the frozen encoder at
224 is 592 MB (inside the ZeroGPU free tier and a T4); encoder latency at 224 on a T4 is 20 ms/image;
the 224 localisation floor is 1/7 of image width per token.

**What this does NOT establish:** whether the zero-shot baseline is good (it is not — 0.0972 / 0.0234
are weak, an ablation floor for the Phase 8 head); whether a **trained** head has the same resolution
sensitivity (re-opening the question after Phase 8 is legitimate *if* the head's validation curve
suggests it, and would be a new pre-registered experiment, not a silent retune); anything about hidden
ISRO/SAC imagery.

---

## 5. Calibration — a measured negative result

**Status:** `MEASURED`; the calibration is **not an improvement**.

Temperature scaling is enabled (`confidence.temperature_scaling: true`) with
`artifacts/calibration_v001.json` (`schema: calibration_v1`). It is scoped to the **R-02 change-VQA
head's answer confidence only** — *other specialists emit their own raw scores and are unaffected*
(`scope.note`).

| Field | Value | Exact key path |
|---|---|---|
| **temperature** | **0.9772731820958189** (≈ 0.9772732) | `temperature_scaling.temperature` |
| fitted on | `Val` | `temperature_scaling.fitted_on` |
| n samples | 16,441 | `temperature_scaling.n_samples` |
| method | `temperature_scaling` | `provenance.method` |
| objective | `mean_negative_log_likelihood` | `provenance.objective` |
| optimizer | `golden_section_on_log_temperature` | `provenance.optimizer` |
| space | `multiclass_logits` | `provenance.space` |
| iterations | 200 | `provenance.iterations` |
| `log_temperature` | −0.022989052824434128 | `fit_diagnostics.log_temperature` |
| `hit_bound` | false | `fit_diagnostics.hit_bound` |
| `effective` | true | `fit_diagnostics.effective` |
| held-out splits excluded | `[Test, Test2]` | `provenance.held_out_splits_excluded` |
| checkpoint sha256 | `cfae5e43b97ca930f568dc5b8ae4f36b24e9ff717af226159802206ffd63a82a` | `provenance.checkpoint_sha256` |

**The result — ECE got worse:**

| Metric | Before | After | Improvement |
|---|---|---|---|
| **ECE** | **0.013755** | **0.014929** | **−0.001174** |
| NLL | 0.689741 | 0.689631 | +0.00011 |

`ece_improvement = −0.001174` is **negative**: expected calibration error **increased**. NLL improved
by a negligible 0.00011 (fit diagnostics record 0.0001103574982127542). `n_bins: 15`,
`n_classes: 19`, `n_samples: 16441`.

**The scaling is retained because it is part of the frozen configuration, not because it helped.** This
is recorded as a negative result, not smoothed over.

**The reliability diagram is explicitly labelled pre-scaling.** `artifacts/calibration_v001.json` →
`reliability_diagram.ece` is **0.013755** (the *before* value), and the block carries the note: *"Equal-
width bins over predicted-class confidence. ECE is bin-count sensitive and is not an aggregate score."*
The diagram on the Benchmark page is labelled **pre-scaling** so a reader cannot mistake it for the
calibrated result. The bin populations range from 8 (bin 0.133–0.2) to 2,814 (bin 0.933–1.0), and two
bins (0.0–0.0667 and 0.0667–0.1333) are empty.

**The consumer contract** (`consumer_contract`): class `TemperatureCalibration` in module
`evidence.confidence`; applied as `sigmoid(logit(z) / T)` for a scalar `z` and `softmax(logits / T)`
for a distribution; read keys `temperature`, `fitted_on|split`, `artifact`, `n_samples`; resolved by
`load_calibration(config, base_dir='configs')`.

**Limitations.** ECE is bin-count sensitive and is not an aggregate score. The calibrated curve is not
plotted — only the pre-scaling diagram is. The temperature is fitted only on the change-VQA head; no
other specialist is calibrated.

---

## 6. Distribution and licensing

### 6.1 Backbones are not redistributed

Every backbone is fetched from the Hugging Face Hub at run time, pinned by revision (§2). No backbone
weight is included in this release, and no backbone is fine-tuned. Their licences are their own — see
each model's HF page:

| Backbone | Repository | Pinned revision |
|---|---|---|
| Router encoder | `sentence-transformers/all-MiniLM-L6-v2` | `1110a243fdf4` |
| VLM | `HuggingFaceTB/SmolVLM-500M-Instruct` | `a7da5b986cb5` |
| Grounding | `chendelong/RemoteCLIP` | `bf1d8a3ccf2d` |
| Optical-SAR | `antofuller/CROMA` | `0dd28e3d633b` |

The change encoder's ResNet-18 trunk is a torchvision model with ImageNet weights; it is likewise not
redistributed.

### 6.2 The six trained artifacts are published by this project

The six trained artifacts (§1) are published on the Hugging Face Hub under `thundercode/SatQuery`
(`models/manifest.json` → `release_repo`), **labelled by kind**, each with its backbone dependency
documented and each accompanied by a checksum:

- [`../models/manifest.json`](../models/manifest.json) — schema `satquery_model_manifest_v1`,
  generated by `release/tools/generate_model_manifest.py`, `artifact_count: 6`,
  `generated_utc: 2026-09-25T18:15:38+00:00`, `config_hash: 78f1e3700da15aa1`. Its `note` states: *"No
  byte count or hash is typed by hand."*
- [`../models/checksums.sha256`](../models/checksums.sha256) — six lines, keyed by the path in the
  repository, verifiable with `sha256sum -c checksums.sha256`.

See [`../HF_RELEASE_VERIFICATION.md`](../HF_RELEASE_VERIFICATION.md) for the upload/verification record.

### 6.3 No licence file exists — OPEN

**There is no `LICENSE` file in the source repository.** This is an **OPEN** item flagged in
[`LIMITATIONS.md`](LIMITATIONS.md) §5 (item 29): a licence must be selected **before any public release
of the code**. Model weights carry the terms of their backbone licences; the six trained artifacts are
distributed under whatever terms the owner selects.

### 6.4 What this release does not contain

- **No backbone weights** (fetched at run time, pinned by revision).
- **No training checkpoints** (archived as provenance, not released — §1.2).
- **No feature caches** (`fusion_features/`, the grounding feature cache, etc. — reproducible, not
  released as weights).
- **No hidden/private evaluation data** (`evaluation.hidden_data_access: false`).

---

## 7. Status summary — what is NOT established

### 7.1 Status of each artifact

| Artifact | Implemented | Measured | Acceptance | Ruling |
|---|---|---|---|---|
| `change` | yes | yes | **VERIFIED** | closed |
| `change_vqa` | yes | yes | MEASURED | **OPEN** |
| `optical_sar` | yes | yes | MEASURED | **OPEN** |
| `grounding` | yes | yes | MEASURED (2 protocols) | Phase 8 complete; head not wired |
| `router` | yes | validation only | MEASURED | **TEST NOT RUN** |
| `vlm` | yes | yes | **ACCEPTANCE-REJECTED** | CLOSED |

### 7.2 NOT RUN / OPEN / BLOCKED for this topic

| Item | State |
|---|---|
| **System-level end-to-end accuracy** | **NOT RUN — none exists**; no such number is claimed anywhere |
| **Router test split** | **NOT RUN** |
| **Grounding resolution at 448 for the trained head** | **NOT RUN** — only the zero-shot baseline was measured at 448; a re-open would be a new pre-registered experiment |
| **`grounding_head` wired into `inference.py`** | **DEFERRED** (Phase 8 integration step) |
| **`change` head wired into serving by default** | **DEFERRED** (registry override, not config) |
| **`change_vqa` serving** | requires both a trained head *and* a trained feature extractor; neither wired by default |
| **`sa_mode: BAM`** | **NOT IMPLEMENTED** (raises) |
| **Router adapter parameter count** | **UNKNOWN** — 50,822 (manifest/docstring) vs 51,725 (Phase 4 report), unreconciled |
| **`optical_sar` / `change` head parameter counts** | `UNKNOWN — not established from the available evidence` (manifest records `null`) |
| **`optical.normalization` / `sar.representation` config keys** | read by **no code** — **OPEN** |
| **CROMA `number_of_patches`** | **UNVERIFIED** upstream; checked against 225 at load time |
| **Grounding validation IoU** | the Phase 8 table records 0.0943 at epoch 20; the prose of the same document rounds it to 0.0946 — the small discrepancy is unreconciled |
| **BigEarthNet multi-label evaluation** | **NOT PRODUCED** — the local subset is single-label |
| **Cross-dataset generalisation** | **NOT RUN** — each specialist is evaluated only on its own training-family test split |
| **`LICENSE` file** | **OPEN** — none exists |
| **B-07 tunnel gaps** | patch prepared, **NOT deployed**. **OPEN** |
| **B-02 `codespace_name` trailing `\n`** | cosmetic; **OPEN** |

### 7.3 Explicit non-claims

- **No claim of state-of-the-art performance** on any benchmark.
- **No claim that the trained heads generalise** beyond their training-family test splits.
- **No claim that calibration improves confidence** — the measured ECE worsened.
- **No claim that the VLM adapter is accepted** for production use.
- **No claim of a system-level accuracy**, because no system-level benchmark exists.
- **No claim that any artifact's accuracy may be quoted without its companion metric** (optical-SAR
  accuracy without macro-F1; grounding under one protocol; change-VQA on one test set).

---

## 8. Evidence index

| Topic | Evidence |
|---|---|
| Manifest + checksums | [`../models/manifest.json`](../models/manifest.json), [`../models/checksums.sha256`](../models/checksums.sha256), `release/tools/generate_model_manifest.py`, `release/tools/model_manifest_report.txt` |
| Model card | [`../MODEL_CARD.md`](../MODEL_CARD.md) |
| HF release verification | [`../HF_RELEASE_VERIFICATION.md`](../HF_RELEASE_VERIFICATION.md) |
| Config registry | `configs/base.yaml`, `core/config.py` |
| `change` | `specialists/change/stanet.py`, `specialists/change/specialist.py`, `artifacts/change/eval_test/eval_result.json` |
| `change_vqa` | `specialists/change/vqa_specialist.py`, `artifacts/change_vqa/run/PROMOTION.json` |
| `optical_sar` | `specialists/optical_sar/{croma,fusion_head,specialist}.py`, `artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json`, `docs/PHASE12_115_METRIC_COMPUTED.md`, `docs/PHASE14_OPTICAL_SAR_DECISIONS.md` |
| `grounding` | `specialists/grounding/{head,remoteclip}.py`, `artifacts/grounding/remoteclip_grounding_v001/eval_result_{canonical,matched6}.json`, `docs/PHASE7_RESOLUTION_DECISION.md`, `docs/PHASE8_GROUNDING_HEAD_DECISION.md` |
| `router` | `router/adapter.py`, `artifacts/router/threshold_sweep_val.json`, `docs/PHASE4_ROUTER_REPORT.md` |
| `vlm` | `specialists/vqa/model.py`, `artifacts/vlm/phase6_closure.json`, `artifacts/vlm/run1_test_recovery/adapter_verification.json`, `docs/PHASE6_CLOSURE.md` |
| Calibration | `artifacts/calibration_v001.json` |
| Cross-links | [`BENCHMARKS.md`](BENCHMARKS.md) (metrics and their rules), [`EVALUATION.md`](EVALUATION.md) (how each number was produced), [`TRAINING.md`](TRAINING.md) (how each head was trained), [`DATASETS.md`](DATASETS.md) (the corpora), [`LIMITATIONS.md`](LIMITATIONS.md) (the honest catalogue), [`REPRODUCIBILITY.md`](REPRODUCIBILITY.md) (how to reproduce) |