Image-to-Text
PyTorch
Safetensors
PEFT
English
remote-sensing
satellite-imagery
earth-observation
change-detection
visual-grounding
image-captioning
visual-question-answering
optical-sar-fusion
sar
multimodal
lora
Instructions to use thundercode/SatQuery with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use thundercode/SatQuery with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
release: add README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,10 @@ result envelope. It runs on CPU, is served from a static frontend, and is live a
|
|
| 15 |
> documented public release of the system, its trained artifacts, and its measured results β
|
| 16 |
> **including the negative ones.**
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
## Table of contents
|
|
@@ -23,18 +27,31 @@ result envelope. It runs on CPU, is served from a static frontend, and is live a
|
|
| 23 |
- [What the system supports](#what-the-system-supports)
|
| 24 |
- [Supported inputs](#supported-inputs)
|
| 25 |
- [Architecture](#architecture)
|
|
|
|
|
|
|
|
|
|
| 26 |
- [Routing and the execution trace](#routing-and-the-execution-trace)
|
|
|
|
| 27 |
- [Real inference vs. the preview path](#real-inference-vs-the-preview-path)
|
| 28 |
- [Models](#models)
|
| 29 |
- [Measured results](#measured-results)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
- [Live validation](#live-validation)
|
|
|
|
|
|
|
| 31 |
- [Installation](#installation)
|
| 32 |
- [Local development](#local-development)
|
| 33 |
- [Deployment](#deployment)
|
|
|
|
| 34 |
- [Reproducibility](#reproducibility)
|
| 35 |
- [Known limitations](#known-limitations)
|
| 36 |
- [Links](#links)
|
| 37 |
- [Citation](#citation)
|
|
|
|
| 38 |
|
| 39 |
---
|
| 40 |
|
|
@@ -57,19 +74,81 @@ Two design rules follow from that, and they are non-negotiable in the codebase:
|
|
| 57 |
segmentation heads; confidence comes from a calibrated scoring path.
|
| 58 |
- **Every result carries an observable execution trace** β never chain-of-thought.
|
| 59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
## What the system supports
|
| 61 |
|
| 62 |
Six specialist tasks. All six are reported `available: true` by the live capability contract
|
| 63 |
-
(`GET /api/capabilities`, probed 2026-09-25).
|
| 64 |
|
| 65 |
-
| Task | What it answers | Assets |
|
| 66 |
-
|---|---|---|
|
| 67 |
-
| `vqa` | A free-form question about a single scene | 1 |
|
| 68 |
-
| `caption` | A description of a single scene | 1 |
|
| 69 |
-
| `grounding` | *Where* is a described object or region β returns boxes | 1 |
|
| 70 |
-
| `change` | *What changed* between two co-registered acquisitions β returns change regions | 2 |
|
| 71 |
-
| `change_vqa` | A yes/no or short question about a detected change | 2 |
|
| 72 |
-
| `optical_sar` | Joint scene classification from an optical + SAR pair | 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
|
| 74 |
## Supported inputs
|
| 75 |
|
|
@@ -88,6 +167,63 @@ when a submitted pair looks like two ordinary photographs rather than an optical
|
|
| 88 |
**Per-file upload limit: 4,194,304 bytes (4 MiB).** Larger files are refused with HTTP 413 β imagery
|
| 89 |
must be downscaled first.
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
## Architecture
|
| 92 |
|
| 93 |
This is the **actually deployed** topology. An older direct-client-to-inference design is superseded.
|
|
@@ -105,10 +241,100 @@ flowchart TD
|
|
| 105 |
RE -->|"tunnel β Render"| B
|
| 106 |
```
|
| 107 |
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
### Repository map
|
| 114 |
|
|
@@ -125,6 +351,97 @@ the Codespace dials out to `POST /tunnel/agent` and holds the connection; Render
|
|
| 125 |
| `artifacts/` | Trained heads, checkpoints, evaluation outputs, provenance |
|
| 126 |
| `docs/` | Architecture, models, benchmarks, deployment, limitations |
|
| 127 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
## Routing and the execution trace
|
| 129 |
|
| 130 |
Routing is deliberately two-stage, and the split matters:
|
|
@@ -136,12 +453,124 @@ Routing is deliberately two-stage, and the split matters:
|
|
| 136 |
decide the task actually dispatched. This is why a reading of `change` with **one** asset dispatches
|
| 137 |
`change_vqa` β the documented quantifier upgrade.
|
| 138 |
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 145 |
|
| 146 |
### The eight execution events
|
| 147 |
|
|
@@ -160,8 +589,18 @@ real network calls (`SQ.EVENT_NAMES` in `frontend/assets/js/core.js`):
|
|
| 160 |
| 8 | `RESULT_ASSEMBLED` | The `ResultEnvelope` is complete |
|
| 161 |
|
| 162 |
These are a **frontend** vocabulary driven by observable events β not a backend protocol and not a
|
| 163 |
-
model's reasoning trace.
|
| 164 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
|
| 166 |
## Real inference vs. the preview path
|
| 167 |
|
|
@@ -175,6 +614,20 @@ marked live; the preview path is the only source of mock-marked nodes.
|
|
| 175 |
The distinction is observable, not asserted: a live run shows `live Β· N evidence Β· transport β¦` and
|
| 176 |
zero `.trace__node.is-mock` elements.
|
| 177 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 178 |
## Models
|
| 179 |
|
| 180 |
Six trained artifacts are released. **Four are task heads and two are adapters** β none is a complete
|
|
@@ -193,6 +646,67 @@ standalone model, and each documents its backbone dependency. Full detail: [`doc
|
|
| 193 |
Backbones are third-party and pinned by `repo_id` + `revision` in `configs/base.yaml`; they are
|
| 194 |
fetched from the Hugging Face Hub, not redistributed here.
|
| 195 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
## Measured results
|
| 197 |
|
| 198 |
Every number below traces to an artifact, a test, or a live run. **Nothing here is a system-level
|
|
@@ -220,10 +734,43 @@ benchmark β no such benchmark exists** (see [Known limitations](#known-limitat
|
|
| 220 |
| Router **overall ungated** accuracy | 0.965116 | val, n=86, corpus-limited | `overall_ungated_accuracy` | **TEST NOT RUN** |
|
| 221 |
| System-level end-to-end benchmark | β | β | β | **NOT RUN β none exists** |
|
| 222 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 223 |
### Grounding: three decode variants, two protocols
|
| 224 |
|
| 225 |
-
The grounding head is evaluated under **two matching protocols** (canonical, matched6) and **three
|
| 226 |
-
decode variants
|
| 227 |
|
| 228 |
| Decode | canonical mean_best_IoU | matched6 mean_best_IoU |
|
| 229 |
|---|---|---|
|
|
@@ -235,6 +782,41 @@ The head clears the zero-shot baseline, but only the threshold decode is meaning
|
|
| 235 |
argmax decode (0.1215) is barely better than zero-shot. The absolute level is modest either way:
|
| 236 |
**grounding is useful, not solved.**
|
| 237 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 238 |
### Fusion: measured but the ruling is open
|
| 239 |
|
| 240 |
Optical-SAR fusion reaches **0.931 accuracy** on a 19-class held-out set of 4,000 β but only
|
|
@@ -242,6 +824,27 @@ Optical-SAR fusion reaches **0.931 accuracy** on a 19-class held-out set of 4,00
|
|
| 242 |
frequent classes and weak on rare ones. The acceptance ruling for this head is **OPEN**, and the
|
| 243 |
headline accuracy must never be quoted without the macro_F1 beside it.
|
| 244 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 245 |
### Change-VQA: two test sets, and they disagree
|
| 246 |
|
| 247 |
`artifacts/change_vqa/run/PROMOTION.json` records **two** test evaluations:
|
|
@@ -254,6 +857,15 @@ headline accuracy must never be quoted without the macro_F1 beside it.
|
|
| 254 |
The `test` numbers are the higher pair. Both are reported here; quoting only `test` would overstate
|
| 255 |
the result. The acceptance ruling is **OPEN**.
|
| 256 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 257 |
### Phase 6 / VLM: deployment success β model acceptance
|
| 258 |
|
| 259 |
The Phase-6 SmolVLM LoRA adapter reaches **exact_match 0.963** and **F1 0.96432** on a frozen
|
|
@@ -262,31 +874,52 @@ The Phase-6 SmolVLM LoRA adapter reaches **exact_match 0.963** and **F1 0.96432*
|
|
| 262 |
Those two verdicts are not in conflict, and the distinction is the point:
|
| 263 |
|
| 264 |
- **USABLE_VERIFIED** β the adapter loads, runs, and produces the measured numbers in the deployed
|
| 265 |
-
pipeline.
|
| 266 |
-
- **ACCEPTANCE-REJECTED** β the change did not clear the project's own pre-registered acceptance bar.
|
|
|
|
|
|
|
| 267 |
|
| 268 |
A model can be a working engineering artifact and a rejected research result at the same time.
|
| 269 |
-
This release keeps both labels.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 270 |
|
| 271 |
### Calibration: it got worse, and we say so
|
| 272 |
|
| 273 |
The `change_vqa` confidence path applies temperature scaling (`T = 0.9773`). Measured on the
|
| 274 |
validation split (n=16441):
|
| 275 |
|
| 276 |
-
| | ECE |
|
| 277 |
-
|---|---|
|
| 278 |
-
| Before temperature scaling | **0.013755** |
|
| 279 |
-
| After temperature scaling | **0.014929** |
|
| 280 |
|
| 281 |
-
**Calibration did not improve β it moved slightly worse.** The
|
|
|
|
| 282 |
of the frozen configuration, not because it helped. The reliability curve plotted on the Benchmark page
|
| 283 |
is explicitly labelled as the **pre-scaling** diagram so a reader cannot mistake it for the calibrated
|
| 284 |
-
result.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
|
| 286 |
## Live validation
|
| 287 |
|
| 288 |
Validation drove the **production site** in a headed browser, one upload per case, with per-case
|
| 289 |
-
screenshots and recorded run identifiers.
|
|
|
|
| 290 |
|
| 291 |
| Property | Result |
|
| 292 |
|---|---|
|
|
@@ -299,13 +932,27 @@ screenshots and recorded run identifiers.
|
|
| 299 |
| Trace fill | 94.4444 % on every live run |
|
| 300 |
| Frontend regression suite | **106 passed** (`tests/unit/test_frontend_live_wiring.py`) |
|
| 301 |
|
| 302 |
-
Each pass produced **fresh run identifiers** β no run id is shared between passes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 303 |
|
| 304 |
The harness asserts the form state **before** dispatch β that the query box really holds the intended
|
| 305 |
query, that `#obsTail` reads `ready`, and that both frames are attached for pair tasks. This matters:
|
| 306 |
an earlier harness revision typed with synthetic key events that Chrome silently drops when the window
|
| 307 |
lacks OS focus, so it dispatched the page's *default* query and still recorded a "result". The
|
| 308 |
-
assertions exist because of that failure.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 309 |
|
| 310 |
### Representative real run IDs
|
| 311 |
|
|
@@ -325,6 +972,12 @@ are fresh on every pass; the other two passes recorded different ids.
|
|
| 325 |
|
| 326 |
The last two are the router-defect queries. Both previously collapsed to `vqa` and answered "River".
|
| 327 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 328 |
### Screenshots
|
| 329 |
|
| 330 |
Eight captures from the post-fix live run (headed browser, 1384Γ855, one upload per case). Each
|
|
@@ -362,6 +1015,29 @@ Backbones are fetched from the Hugging Face Hub on first use, pinned by revision
|
|
| 362 |
`configs/base.yaml`. The frozen config hash is **`78f1e3700da15aa1`** β the loader refuses to run a
|
| 363 |
config that violates the recorded invariants (for example `fusion.input_dim == 3*encoder_dim + 12 + 2`).
|
| 364 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 365 |
## Local development
|
| 366 |
|
| 367 |
```bash
|
|
@@ -384,6 +1060,33 @@ Run the frontend regression suite:
|
|
| 384 |
python -m pytest tests/unit/test_frontend_live_wiring.py -q
|
| 385 |
```
|
| 386 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 387 |
## Deployment
|
| 388 |
|
| 389 |
The live topology is **Cloudflare Pages β Render β outbound tunnel β GitHub Codespace**.
|
|
@@ -399,25 +1102,94 @@ Deployment sources are **separate repositories** from this release. The wake flo
|
|
| 399 |
Render β start the Codespace if stopped β poll `/v1/health` β surface *"Waking inference engineβ¦"* β
|
| 400 |
`POST /infer` β result.
|
| 401 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 402 |
### Deployment caveats
|
| 403 |
|
| 404 |
- **Cold start.** The inference host may be stopped when idle. The first request after a cold start
|
| 405 |
can exceed the client timeout while weights are fetched; a retry a few seconds later normally
|
| 406 |
succeeds. Warm the stack before any demonstration and confirm
|
| 407 |
-
`GET /api/health` reports `tunnel.agent_connected: true`.
|
|
|
|
| 408 |
- **Tunnel gaps.** The tunnel agent can be briefly absent. A request issued during such a gap may hang
|
| 409 |
or return HTTP 504. **This is not fixed in production** β a prepared patch
|
| 410 |
(`forward_unavailable` 503 / `upstream_timeout` 504 plus a `codespace_name` fix) exists and is
|
| 411 |
documented, but it was deliberately not deployed. Root cause: in `auto` transport mode a tunnel
|
| 412 |
-
timeout falls through to the forwarded-port path, which then spends the 120 s wake
|
| 413 |
-
HTTP 302 β the observed ~249 s failure.
|
| 414 |
- **`codespace_name`** is still reported with a trailing newline by `/api/health` (cosmetic; the wake
|
| 415 |
path strips it).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 416 |
|
| 417 |
## Reproducibility
|
| 418 |
|
| 419 |
1. **Configuration.** `configs/base.yaml` is the single registry; no magic numbers in Python. Its hash
|
| 420 |
-
is recorded in every execution trace. Frozen hash: `78f1e3700da15aa1`.
|
|
|
|
| 421 |
2. **Backbones.** Pinned by `repo_id` + `revision`, never by floating tag:
|
| 422 |
`HuggingFaceTB/SmolVLM-500M-Instruct@a7da5b986cb5`,
|
| 423 |
`chendelong/RemoteCLIP@bf1d8a3ccf2d`,
|
|
@@ -430,6 +1202,62 @@ Render β start the Codespace if stopped β poll `/v1/health` β surface *"Wa
|
|
| 430 |
5. **Prompts** are versioned files, frozen before benchmark evaluation.
|
| 431 |
6. **Negative results are preserved.** Rejected and open rulings are recorded, not removed.
|
| 432 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 433 |
## Known limitations
|
| 434 |
|
| 435 |
1. **No system-level end-to-end benchmark exists.** Per-specialist metrics are real; a single
|
|
@@ -439,20 +1267,38 @@ Render β start the Codespace if stopped β poll `/v1/health` β surface *"Wa
|
|
| 439 |
modality accounting in the response confirms the right channels reached the fusion head, but the
|
| 440 |
presentation is not user-facing.
|
| 441 |
4. **VQA is weak-but-related.** Asked what terrain dominates a scene, it answers "Grassland".
|
| 442 |
-
5. **Fusion macro_F1 is low (0.
|
| 443 |
-
|
| 444 |
-
|
| 445 |
-
|
|
|
|
|
|
|
| 446 |
8. **Router lexical residuals.** *"What is the new runway?"* reads `change` rather than `vqa` (the
|
| 447 |
`new`-as-change heuristic fires outside `where` questions), and *"How much built-up area was
|
| 448 |
added?"* reads `vqa` (under-trigger). A lexical router cannot cleanly separate "the new X" from
|
| 449 |
"what's new"; a trained intent router exists in `artifacts/router/` but is not attached.
|
| 450 |
-
9. **B-07 tunnel gaps are not fixed in production** (see Deployment caveats).
|
| 451 |
10. **No license has been selected** for this repository. Until one is, the artifacts carry
|
| 452 |
`license: unknown` and no reuse rights should be assumed. This is an open owner decision.
|
| 453 |
11. **The Anatomy of a Run page** renders a recorded run whose plate uses the 720Γ720 variant of an
|
| 454 |
image analysed at 730Γ730 β identical content, scaled by the canvas, but the "actual analysed
|
| 455 |
image" wording is slightly loose.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 456 |
|
| 457 |
## Links
|
| 458 |
|
|
@@ -472,7 +1318,35 @@ Third-party models this work builds on (pinned, not redistributed):
|
|
| 472 |
| [`antofuller/CROMA`](https://huggingface.co/antofuller/CROMA) | `0dd28e3d633b` | optical/SAR fusion encoder |
|
| 473 |
|
| 474 |
Datasets referenced by the evaluations: LEVIR-CD-256 (change), VRSBench (grounding),
|
| 475 |
-
BigEarthNet (optical-SAR fusion, 19 CLC classes). No dataset is
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 476 |
|
| 477 |
## Citation
|
| 478 |
|
|
@@ -491,3 +1365,11 @@ No paper accompanies this release. Until one exists, cite the repository:
|
|
| 491 |
## License
|
| 492 |
|
| 493 |
**Not yet selected.** See limitation 10. Backbone models remain under their own upstream licenses.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
> documented public release of the system, its trained artifacts, and its measured results β
|
| 16 |
> **including the negative ones.**
|
| 17 |
|
| 18 |
+
This README is the front door to a long-form documentation set. It is written to the same standard as
|
| 19 |
+
the rest of the release: **every number, path, run identifier and status is taken from a file that was
|
| 20 |
+
actually read**, and where something was never run, that is stated rather than implied.
|
| 21 |
+
|
| 22 |
---
|
| 23 |
|
| 24 |
## Table of contents
|
|
|
|
| 27 |
- [What the system supports](#what-the-system-supports)
|
| 28 |
- [Supported inputs](#supported-inputs)
|
| 29 |
- [Architecture](#architecture)
|
| 30 |
+
- [Repository map](#repository-map)
|
| 31 |
+
- [The four endpoints](#the-four-endpoints)
|
| 32 |
+
- [Documentation map](#documentation-map)
|
| 33 |
- [Routing and the execution trace](#routing-and-the-execution-trace)
|
| 34 |
+
- [The eight execution events](#the-eight-execution-events)
|
| 35 |
- [Real inference vs. the preview path](#real-inference-vs-the-preview-path)
|
| 36 |
- [Models](#models)
|
| 37 |
- [Measured results](#measured-results)
|
| 38 |
+
- [Grounding: three decode variants, two protocols](#grounding-three-decode-variants-two-protocols)
|
| 39 |
+
- [Fusion: measured but the ruling is open](#fusion-measured-but-the-ruling-is-open)
|
| 40 |
+
- [Change-VQA: two test sets, and they disagree](#change-vqa-two-test-sets-and-they-disagree)
|
| 41 |
+
- [Phase 6 / VLM: deployment success β model acceptance](#phase-6--vlm-deployment-success--model-acceptance)
|
| 42 |
+
- [Calibration: it got worse, and we say so](#calibration-it-got-worse-and-we-say-so)
|
| 43 |
- [Live validation](#live-validation)
|
| 44 |
+
- [Representative real run IDs](#representative-real-run-ids)
|
| 45 |
+
- [Screenshots](#screenshots)
|
| 46 |
- [Installation](#installation)
|
| 47 |
- [Local development](#local-development)
|
| 48 |
- [Deployment](#deployment)
|
| 49 |
+
- [Deployment caveats](#deployment-caveats)
|
| 50 |
- [Reproducibility](#reproducibility)
|
| 51 |
- [Known limitations](#known-limitations)
|
| 52 |
- [Links](#links)
|
| 53 |
- [Citation](#citation)
|
| 54 |
+
- [License](#license)
|
| 55 |
|
| 56 |
---
|
| 57 |
|
|
|
|
| 74 |
segmentation heads; confidence comes from a calibrated scoring path.
|
| 75 |
- **Every result carries an observable execution trace** β never chain-of-thought.
|
| 76 |
|
| 77 |
+
### Why not one end-to-end model
|
| 78 |
+
|
| 79 |
+
The design is a *router plus specialists*, not a single model that "does satellite QA". Each layer has
|
| 80 |
+
exactly one verb, assigned in the architecture freeze:
|
| 81 |
+
|
| 82 |
+
> router **understands**; policy engine **decides**; specialists **compute**; VLM **explains**;
|
| 83 |
+
> evidence engine **proves**.
|
| 84 |
+
|
| 85 |
+
That division is not decoration β it resolves real ambiguities about *where* a decision belongs. The
|
| 86 |
+
`Intent` type makes the first row explicit in code:
|
| 87 |
+
|
| 88 |
+
```python
|
| 89 |
+
class Intent(BaseModel):
|
| 90 |
+
"""Output of the learned router. Advisory only β the controller decides."""
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
The four constraints that force the modular design:
|
| 94 |
+
|
| 95 |
+
| Constraint | Consequence |
|
| 96 |
+
|---|---|
|
| 97 |
+
| The system must run on **CPU** | End-to-end VLM inference at usable quality needs a GPU; small per-task modules do not. |
|
| 98 |
+
| Different tasks have **incompatible outputs** | `change` returns a spatial change map; `caption` returns prose; `optical_sar` returns a class distribution. One head cannot emit all three. |
|
| 99 |
+
| Tasks have **different data and metrics** | Each specialist is trained and evaluated on its own split with its own protocol. |
|
| 100 |
+
| **Truthfulness** | Per-task metrics are auditable. A single end-to-end number would hide which component failed. |
|
| 101 |
+
|
| 102 |
+
The cost of this design is that there is **no system-level accuracy number** β because there is no
|
| 103 |
+
single model to measure. That absence is stated rather than papered over; see
|
| 104 |
+
[Known limitations](#known-limitations) item 1.
|
| 105 |
+
|
| 106 |
+
### What the system is deliberately not
|
| 107 |
+
|
| 108 |
+
| Absent | Why |
|
| 109 |
+
|---|---|
|
| 110 |
+
| Database, authentication, users, job queue | the gateway is stateless by design; inference is synchronous |
|
| 111 |
+
| GPU requirement | device is selected via `SATQUERY_DEVICE`; all placement is `.to(device)`, never `.cuda()` |
|
| 112 |
+
| Gradio GUI | the frontend is a separate static tier; `app/space_app.py` serves JSON only |
|
| 113 |
+
| Chain-of-thought in traces | traces carry observable facts only β states, timings, counts, config hash, model refs |
|
| 114 |
+
| Backbone redistribution | backbones are fetched from the Hugging Face Hub, pinned by revision |
|
| 115 |
+
| System-level end-to-end benchmark | **NOT RUN β none exists** |
|
| 116 |
+
| Router test-split evaluation | **NOT RUN** |
|
| 117 |
+
|
| 118 |
## What the system supports
|
| 119 |
|
| 120 |
Six specialist tasks. All six are reported `available: true` by the live capability contract
|
| 121 |
+
(`GET /api/capabilities`, probed 2026-09-25, `schema_version 1.0`).
|
| 122 |
|
| 123 |
+
| Task | What it answers | Assets | `requires_pair` | `max_assets` |
|
| 124 |
+
|---|---|---|---|---|
|
| 125 |
+
| `vqa` | A free-form question about a single scene | 1 | false | 1 |
|
| 126 |
+
| `caption` | A description of a single scene | 1 | false | 1 |
|
| 127 |
+
| `grounding` | *Where* is a described object or region β returns boxes | 1 | false | 1 |
|
| 128 |
+
| `change` | *What changed* between two co-registered acquisitions β returns change regions | 2 | true | 2 |
|
| 129 |
+
| `change_vqa` | A yes/no or short question about a detected change | 2 | true | 2 |
|
| 130 |
+
| `optical_sar` | Joint scene classification from an optical + SAR pair | 2 | true | 2 |
|
| 131 |
+
|
| 132 |
+
The live contract also carries per-task notes β `vqa`/`caption` fetch SmolVLM weights from the Hub on
|
| 133 |
+
first use, `grounding` fetches the RemoteCLIP encoder on first use, and `optical_sar` declares
|
| 134 |
+
`modalities: ["optical", "sar"]`.
|
| 135 |
+
|
| 136 |
+
### The ontology is wider than the capability list
|
| 137 |
+
|
| 138 |
+
There are **two** related vocabularies, and they are not the same six:
|
| 139 |
+
|
| 140 |
+
- `core/schemas.py::Task` carries **seven** values: `vqa`, `caption`, `grounding`, `change`,
|
| 141 |
+
`optical_sar`, `change_vqa`, `unsupported`.
|
| 142 |
+
- The router's label space (`router/label_space.py::TASK_CLASSES`) is **six** classes:
|
| 143 |
+
`vqa, caption, grounding, change, optical_sar, unsupported`.
|
| 144 |
+
- `GET /api/capabilities` lists **six** tasks β the same six as the router's *minus* `unsupported`,
|
| 145 |
+
*plus* `change_vqa`.
|
| 146 |
+
|
| 147 |
+
This asymmetry is intentional. `unsupported` is a **routing outcome** ("this is not a satellite-imagery
|
| 148 |
+
question"), not a servable capability. `change_vqa` is reached through the change family rather than
|
| 149 |
+
being a separate router class, and it is servable. The three sets are reconciled in one place β the
|
| 150 |
+
frontend's `ROUTER_TASK_TO_SERVER` map (`frontend/assets/js/mission.js`) β because `AnalysisRequest` is
|
| 151 |
+
`extra="forbid"` and any string outside the `Task` enum is a 422.
|
| 152 |
|
| 153 |
## Supported inputs
|
| 154 |
|
|
|
|
| 167 |
**Per-file upload limit: 4,194,304 bytes (4 MiB).** Larger files are refused with HTTP 413 β imagery
|
| 168 |
must be downscaled first.
|
| 169 |
|
| 170 |
+
### The size cap is one number shared by two layers
|
| 171 |
+
|
| 172 |
+
The cap is not a literal in two places; both the gateway and the inference service read
|
| 173 |
+
`SATQUERY_MAX_FILE_BYTES`, and the default is `4 * 1024 * 1024` in both. The inference service's
|
| 174 |
+
`_asset_max_file_bytes()` (`app/space_app.py`) is deliberately strict about it:
|
| 175 |
+
|
| 176 |
+
- a **missing** variable β the 4 MiB default;
|
| 177 |
+
- a **non-integer** value β `ValueError` (not silently defaulted);
|
| 178 |
+
- a **non-positive** value β `ValueError` (a cap of `0` refuses every upload, which is a configuration
|
| 179 |
+
error rather than a limit).
|
| 180 |
+
|
| 181 |
+
The reason is recorded in the source: a silent default would let a deployment whose operator typed a
|
| 182 |
+
malformed cap keep accepting uploads against a limit nobody chose, while the gateway refused to boot
|
| 183 |
+
for the *same* value β the two layers disagreeing about what "too large" means, which is exactly the
|
| 184 |
+
failure the shared variable exists to prevent.
|
| 185 |
+
|
| 186 |
+
The accepted content types mirror the gateway's allowlist (defence in depth β the Space validates
|
| 187 |
+
independently rather than trusting the gateway to be its only caller):
|
| 188 |
+
|
| 189 |
+
```
|
| 190 |
+
image/tiff, image/geotiff, image/png, image/jpeg, application/octet-stream
|
| 191 |
+
```
|
| 192 |
+
|
| 193 |
+
### Uploads are handle-based, and the Space owns the bytes
|
| 194 |
+
|
| 195 |
+
`POST /v1/assets` accepts one file and returns an **opaque ephemeral handle**. The store lives on the
|
| 196 |
+
inference host, not the gateway, because the inference host is where `inspect_raster` reads the bytes
|
| 197 |
+
and where `cache_max_models: 1` serialises their consumption β a gateway-side store would put the bytes
|
| 198 |
+
on a different machine from the reader. The upload endpoint is **off by default** and enabled only when
|
| 199 |
+
`SATQUERY_ASSET_ENABLED` and `SATQUERY_ASSET_DIR` are both set; otherwise it answers a named
|
| 200 |
+
`model_unavailable` envelope explaining the switch. Handle capacity defaults to **32** and the handle
|
| 201 |
+
TTL to **900.0 s**, both overridable by environment (and read from the environment rather than
|
| 202 |
+
`configs/base.yaml` on purpose β adding a key there would move the frozen config hash).
|
| 203 |
+
|
| 204 |
+
### Input geometry and normalisation
|
| 205 |
+
|
| 206 |
+
Downstream of the format check, input handling is governed by the frozen registry
|
| 207 |
+
(`configs/base.yaml`):
|
| 208 |
+
|
| 209 |
+
| Key | Value | Meaning |
|
| 210 |
+
|---|---|---|
|
| 211 |
+
| `image.max_pixels` | 25,000,000 | hard ceiling on decoded pixels |
|
| 212 |
+
| `image.tile_size` | 512 | tile edge |
|
| 213 |
+
| `image.tile_overlap` | 128 | tile stride overlap |
|
| 214 |
+
| `image.max_tiles` | 64 | hard ceiling on tiles *examined* |
|
| 215 |
+
| `image.top_k_tiles` | 4 | tiles actually sent through a specialist |
|
| 216 |
+
| `optical.normalization` | percentile | 2ndβ98th percentile stretch |
|
| 217 |
+
| `optical.lower_percentile` / `upper_percentile` | 2 / 98 | stretch bounds |
|
| 218 |
+
| `optical.canonical_channels` | 12 | CROMA expects exactly 12 optical channels |
|
| 219 |
+
| `sar.representation` | db | SAR is converted to decibels |
|
| 220 |
+
| `sar.clip_min_db` / `clip_max_db` | β30 / 5 | dB clip window |
|
| 221 |
+
| `sar.canonical_channels` | 2 | CROMA expects exactly 2 SAR channels (VV, VH) |
|
| 222 |
+
|
| 223 |
+
The tile policy follows the plan's section 9.1: whole-image thumbnail first, then top-K tiles. `max_tiles`
|
| 224 |
+
is the hard ceiling on tiles examined; `top_k_tiles` is how many are actually dispatched β and the loader
|
| 225 |
+
**rejects** a config where `top_k_tiles > max_tiles`.
|
| 226 |
+
|
| 227 |
## Architecture
|
| 228 |
|
| 229 |
This is the **actually deployed** topology. An older direct-client-to-inference design is superseded.
|
|
|
|
| 241 |
RE -->|"tunnel β Render"| B
|
| 242 |
```
|
| 243 |
|
| 244 |
+
### Why a tunnel
|
| 245 |
+
|
| 246 |
+
The inference host runs in a GitHub Codespace. The forwarded-port path is not reachable for a private
|
| 247 |
+
repo (it returns HTTP 302), so the orchestrator keeps a **long-poll tunnel**: the Codespace dials out
|
| 248 |
+
to `POST /tunnel/agent` and holds the connection; Render queues work onto it. `transport_mode` is
|
| 249 |
+
`auto`, and the tunnel is the live transport. There is **no** `SATQUERY_UPSTREAM_URL` and **no**
|
| 250 |
+
`HF_TOKEN` in the live configuration β the transport is the outbound tunnel, not a forwarded port.
|
| 251 |
+
|
| 252 |
+
The live health payload (`GET /api/health`, probed 2026-09-25) records the tunnel's state directly:
|
| 253 |
+
|
| 254 |
+
```json
|
| 255 |
+
{"status":"ok","service":"satquery-orchestrator",
|
| 256 |
+
"tunnel":{"agent_connected":true,"agent_id":"codespaces-fd1038","pending":0,"completed":97},
|
| 257 |
+
"config":{"codespace_name":"potential-space-trout-r4ppw969w45j2pvvw\n","codespace_port":8000,
|
| 258 |
+
"transport_mode":"auto","tunnel_timeout_s":150.0,"wake_timeout_s":120.0,
|
| 259 |
+
"upstream_timeout_s":90.0,"device":"cpu","has_github_token":true}}
|
| 260 |
+
```
|
| 261 |
+
|
| 262 |
+
Note `codespace_name` still carries a trailing `\n` β that is item B-02, cosmetic, and
|
| 263 |
+
[still open](#deployment-caveats).
|
| 264 |
+
|
| 265 |
+
### The nine-state controller
|
| 266 |
+
|
| 267 |
+
The inference host is a FastAPI service built by `build_space_app()`. Behind the transport sits a
|
| 268 |
+
deterministic controller with a **nine-state** finite state machine (`core/schemas.py::ControllerState`,
|
| 269 |
+
mirrored in `configs/base.yaml` Β§`agent.states`):
|
| 270 |
+
|
| 271 |
+
```
|
| 272 |
+
RECEIVE β PARSE β VALIDATE β PLAN β PREPROCESS β EXECUTE β AGGREGATE β VERIFY β RESPOND
|
| 273 |
+
```
|
| 274 |
+
|
| 275 |
+
The controller is the **only** component that dispatches. `agent.max_specialists` is 4,
|
| 276 |
+
`agent.timeout_seconds` is 120, and `agent.unload_after_workflow` is true β the controller unloads
|
| 277 |
+
models after a workflow so that `cache_max_models: 1` is honoured rather than thrashing the cache.
|
| 278 |
+
|
| 279 |
+
### Frozen backbones, trained modules
|
| 280 |
+
|
| 281 |
+
Four backbones are pinned by `repo_id` + `revision` and fetched from the Hub on first use. Nothing is
|
| 282 |
+
fine-tuned end-to-end.
|
| 283 |
+
|
| 284 |
+
| Role | Repository | Revision | Size / notes |
|
| 285 |
+
|---|---|---|---|
|
| 286 |
+
| Router encoder | `sentence-transformers/all-MiniLM-L6-v2` | `1110a243fdf4` | 90.9 MB, 22,713,216 params, 384-dim embeddings |
|
| 287 |
+
| VLM | `HuggingFaceTB/SmolVLM-500M-Instruct` | `a7da5b986cb5` | ~1015 MB safetensors |
|
| 288 |
+
| Grounding | `chendelong/RemoteCLIP` (`RemoteCLIP-ViT-B-32.pt`) | `bf1d8a3ccf2d` | 605.2 MB; width 768, **projected** dim 512 |
|
| 289 |
+
| Optical-SAR | `antofuller/CROMA` (`CROMA_base.pt`) | `0dd28e3d633b` | 777.6 MB; `encoder_dim` 768, `image_resolution` 120 |
|
| 290 |
+
|
| 291 |
+
Two consequences follow: the system is small (the six trained artifacts total ~125 MiB; everything else
|
| 292 |
+
is public weights), and **no backbone weights are redistributed** by this release.
|
| 293 |
+
|
| 294 |
+
### The evidence and confidence stages
|
| 295 |
+
|
| 296 |
+
Specialists each emit `Evidence` for what *they* computed. The `EvidenceEngine`
|
| 297 |
+
(`evidence/engine.py`) does not re-derive any of it; it aggregates:
|
| 298 |
+
|
| 299 |
+
```
|
| 300 |
+
collect across specialists β deduplicate β order β renumber β cap
|
| 301 |
+
```
|
| 302 |
+
|
| 303 |
+
The pipeline is **pure and deterministic** β no clock, no RNG, no I/O β and its purity is a test
|
| 304 |
+
assertion, not a hope. Three details matter:
|
| 305 |
+
|
| 306 |
+
- **Stable identity.** `Evidence.evidence_id` defaults to a random uuid, which is useless across runs.
|
| 307 |
+
The engine renumbers to `evidence_001`, `evidence_002`, β¦ (zero-padded to three digits, well past the
|
| 308 |
+
`evidence.max_items` bound of 32) so a downstream artefact can cite one item deterministically.
|
| 309 |
+
- **Defined order.** Items are sorted by `(type, source_specialist, score DESC, coordinates)`. The
|
| 310 |
+
coordinate tie-breaker is what makes the order *total*; without it, two items sharing type,
|
| 311 |
+
specialist and score would fall back to Python's stable-sort insertion order, reintroducing
|
| 312 |
+
input-order dependence.
|
| 313 |
+
- **Content identity, not container identity.** Dedup keys on
|
| 314 |
+
`(type, source_specialist, coordinate_system, rounded coordinates, rounded score)` β `payload`,
|
| 315 |
+
`artifact_ref` and `evidence_id` are deliberately excluded. Two items that agree on the same
|
| 316 |
+
geolocation, one carrying a `crs` and one not, have made the same claim about the world; the surviving
|
| 317 |
+
item's payload is merged with the discarded one's so the `crs` is not lost. Two items that share a type
|
| 318 |
+
and score but **disagree** on coordinates are two different claims and are both kept β suppressing a
|
| 319 |
+
spatial disagreement would be the silent contradiction the freeze forbids.
|
| 320 |
+
|
| 321 |
+
The engine records `dropped_duplicates` (non-zero is normal and healthy β it means two specialists
|
| 322 |
+
agreed), `dropped_over_limit` (non-zero is a warning β a specialist's evidence did not survive), and
|
| 323 |
+
`truncated`. `evidence_digest()` exists so that reproducibility is an assertion:
|
| 324 |
+
|
| 325 |
+
```
|
| 326 |
+
aggregate(inputs_a) is reproducible iff digest(a) == digest(b)
|
| 327 |
+
```
|
| 328 |
+
|
| 329 |
+
The confidence stage is `evidence/confidence.py`, and it exists to enforce one rule: **a calibration that
|
| 330 |
+
claims to be fitted when it is not is a false claim of reliability.** So when no fitted artifact is
|
| 331 |
+
available it passes the raw score through unchanged, sets `method="uncalibrated"`, and leaves
|
| 332 |
+
`calibrated=None` β which is what makes it honest, because `ConfidenceBreakdown.value` then returns
|
| 333 |
+
`raw`, and any consumer can distinguish "we calibrated this" from "we did not". The temperature scaling
|
| 334 |
+
itself is `calibrated = sigmoid(logit(z) / T)`, with two stated failure modes handled explicitly: a
|
| 335 |
+
fitted `T` of exactly `1.0` is the identity map and is reported as uncalibrated rather than silently
|
| 336 |
+
pretending to have done something, and `z = 0.0` (whose log-odds diverge) is clamped at the boundary so
|
| 337 |
+
a hard zero cannot become a NaN.
|
| 338 |
|
| 339 |
### Repository map
|
| 340 |
|
|
|
|
| 351 |
| `artifacts/` | Trained heads, checkpoints, evaluation outputs, provenance |
|
| 352 |
| `docs/` | Architecture, models, benchmarks, deployment, limitations |
|
| 353 |
|
| 354 |
+
The component inventory in full (every path is the authoritative location):
|
| 355 |
+
|
| 356 |
+
| Layer | Module | Responsibility |
|
| 357 |
+
|---|---|---|
|
| 358 |
+
| **Contracts** | `core/schemas.py` | the binding typed contract: `Task`, `Intent`, `Evidence`, `SpecialistResult`, `ResultEnvelope`, `ExecutionTrace`, β¦ |
|
| 359 |
+
| **Config** | `core/config.py` | load, deep-merge, validate, hash the registry; `get_config()` singleton |
|
| 360 |
+
| **Errors** | `core/errors.py` | the error taxonomy (`ConfigError`, `ModelLoadError`, `RoutingError`, `WorkflowPlanError`, β¦) |
|
| 361 |
+
| **Planning** | `core/planner.py` | turn an `Intent` into a concrete, ordered `ExecutionPlan` |
|
| 362 |
+
| **Registry** | `core/registry.py` | specialist registration / lookup |
|
| 363 |
+
| **Controller** | `core/controller.py` | the nine-state FSM; the only thing that dispatches |
|
| 364 |
+
| **Router** | `router/encoder.py` | frozen MiniLM embedding, cached |
|
| 365 |
+
| | `router/adapter.py` | the five-head `IntentAdapter` (the only trainable router part) |
|
| 366 |
+
| | `router/classifier.py` | learned classification + confidence gate + fallback selection |
|
| 367 |
+
| | `router/fallback.py` | deterministic lexical fallback (`lexical_route`) |
|
| 368 |
+
| | `router/label_space.py` | the ontology, single source of truth |
|
| 369 |
+
| | `router/dataset.py`, `router/train.py` | dataset generation and training |
|
| 370 |
+
| **Specialists** | `specialists/base.py` | the specialist interface |
|
| 371 |
+
| | `specialists/vqa/{model,inference,prompts}.py` | SmolVLM VQA |
|
| 372 |
+
| | `specialists/grounding/{remoteclip,head,inference,specialist}.py` | RemoteCLIP + head |
|
| 373 |
+
| | `specialists/change/{stanet,specialist,postprocess,vqa_specialist}.py` | change detection + change-VQA |
|
| 374 |
+
| | `specialists/optical_sar/{croma,fusion_head,inference,specialist,sensor_adapter,radiometry,prompts}.py` | CROMA fusion |
|
| 375 |
+
| **Evidence** | `evidence/engine.py` | aggregation: dedup β sort β renumber β cap |
|
| 376 |
+
| | `evidence/confidence.py` | temperature scaling, honest pass-through |
|
| 377 |
+
| **Inference app** | `app/space_app.py` | `build_space_app()`; the four-endpoint JSON contract |
|
| 378 |
+
| | `app/serving.py` | the composition root (`build_serving_controller()`) |
|
| 379 |
+
| | `app/deployment.py` | capability / health payload builders |
|
| 380 |
+
| **Gateway** | `gateway/` | the Render orchestrator, asset store, policy/error translation |
|
| 381 |
+
| **Frontend** | `frontend/` | the static site |
|
| 382 |
+
|
| 383 |
+
### The four endpoints
|
| 384 |
+
|
| 385 |
+
The inference service serves exactly **four** JSON endpoints (`app/space_app.py`). No Gradio GUI exists
|
| 386 |
+
in code; the file serves JSON only, by design, so it cannot compete with the static frontend's contract.
|
| 387 |
+
|
| 388 |
+
| Endpoint | Method | Purpose |
|
| 389 |
+
|---|---|---|
|
| 390 |
+
| `/v1/health` | GET | liveness + capability states; **loads no model** |
|
| 391 |
+
| `/v1/capabilities` | GET | the six servable tasks with `requires_pair` / `max_assets` |
|
| 392 |
+
| `/v1/assets` | POST | accept one uploaded file, return an opaque ephemeral handle |
|
| 393 |
+
| `/v1/analyze` | POST | run one analysis; returns a `ResultEnvelope` |
|
| 394 |
+
|
| 395 |
+
The gateway in front of it exposes the same functionality under `/api/*` and holds the request-side
|
| 396 |
+
security boundary. Two details are worth recording because they cost real debugging time:
|
| 397 |
+
|
| 398 |
+
- **Framework-raised errors carry the same envelope.** An unmatched route (404) and a method mismatch
|
| 399 |
+
(405) are wrapped by a Starlette exception handler so they answer with the contract's error shape
|
| 400 |
+
(`routing_error`, `recoverable: false`) rather than FastAPI's default `{"detail": β¦}`. An unhandled
|
| 401 |
+
exception answers with `satquery_error` and a fixed, operator-safe message; the exception's own text is
|
| 402 |
+
logged **server-side only**, so a traceback cannot disclose internal paths to an unauthenticated caller.
|
| 403 |
+
- **The upload body is read bounded.** `POST /v1/assets` uses `read_body_bounded(request, cap)` rather
|
| 404 |
+
than `await request.body()`, so the cap is applied *while reading* rather than after the whole body has
|
| 405 |
+
been buffered. The measured defect this fixed: with the cap at 1 MiB, a 64 MiB body produced a peak
|
| 406 |
+
allocation of 128 MiB, tracking body size linearly with no ceiling, and the `413` came only after
|
| 407 |
+
everything had been held.
|
| 408 |
+
|
| 409 |
+
### Documentation map
|
| 410 |
+
|
| 411 |
+
The architecture reference is a hub plus ten deep sub-documents. Every one of them is written at
|
| 412 |
+
long-form depth, with real signatures, schemas, numbers and file paths.
|
| 413 |
+
|
| 414 |
+
| # | Document | What it covers |
|
| 415 |
+
|---|---|---|
|
| 416 |
+
| β | [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | the architecture hub: thesis, sub-document index, cross-cutting principles, what is deliberately absent |
|
| 417 |
+
| 01 | [`docs/architecture/01-system-overview.md`](docs/architecture/01-system-overview.md) | the thesis, the component inventory, the frozen-backbone strategy, what is deliberately absent |
|
| 418 |
+
| 02 | [`docs/architecture/02-deployment-topology.md`](docs/architecture/02-deployment-topology.md) | the four tiers, the gateway, the outbound tunnel, wake flow, cold start, `transport_mode` |
|
| 419 |
+
| 03 | [`docs/architecture/03-request-lifecycle.md`](docs/architecture/03-request-lifecycle.md) | the nine-state controller, validation rules, modality inference, tiling |
|
| 420 |
+
| 04 | [`docs/architecture/04-router.md`](docs/architecture/04-router.md) | frozen MiniLM, the five-head adapter, `interpret()` vs `chooseTask()`, the lexical fallback, the label space |
|
| 421 |
+
| 05 | [`docs/architecture/05-specialists.md`](docs/architecture/05-specialists.md) | all six tasks: entry points, preprocessing, postprocessing, outputs |
|
| 422 |
+
| 06 | [`docs/architecture/06-evidence-and-confidence.md`](docs/architecture/06-evidence-and-confidence.md) | the evidence schema, the aggregation pipeline, temperature scaling, the eight execution events |
|
| 423 |
+
| 07 | [`docs/architecture/07-configuration-freeze.md`](docs/architecture/07-configuration-freeze.md) | the registry, the enforced invariants, the config hash, why it is frozen |
|
| 424 |
+
| 08 | [`docs/architecture/08-api-contract.md`](docs/architecture/08-api-contract.md) | the four endpoints, the envelopes, error codes, transport headers |
|
| 425 |
+
| 09 | [`docs/architecture/09-frontend.md`](docs/architecture/09-frontend.md) | the static pages, the Analyze console, real-vs-preview, platform traps |
|
| 426 |
+
| 10 | [`docs/architecture/10-observability-and-ops.md`](docs/architecture/10-observability-and-ops.md) | health, counters, traces, what is and is not observed |
|
| 427 |
+
|
| 428 |
+
Companion references, all in this repository:
|
| 429 |
+
|
| 430 |
+
| Document | Contents |
|
| 431 |
+
|---|---|
|
| 432 |
+
| [`docs/MODELS.md`](docs/MODELS.md) | the six artifacts in detail, backbone pinning, rejected model decisions |
|
| 433 |
+
| [`MODEL_CARD.md`](MODEL_CARD.md) | the Hugging Face model card (intended use, out-of-scope use, measured performance) |
|
| 434 |
+
| [`models/manifest.json`](models/manifest.json) | machine-generated byte counts and sha256, one entry per artifact |
|
| 435 |
+
| [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | the headline metric table and the rules it follows |
|
| 436 |
+
| [`docs/EVALUATION.md`](docs/EVALUATION.md) | how each number was produced; evaluation-honesty rules; behavioural validation |
|
| 437 |
+
| [`docs/DATASETS.md`](docs/DATASETS.md) | LEVIR-CD-256, VRSBench, BigEarthNet, CDVQA/SECOND β measured corpus figures |
|
| 438 |
+
| [`docs/TRAINING.md`](docs/TRAINING.md) | per-artifact hyperparameters and where each was trained |
|
| 439 |
+
| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | live revisions, env vars, deploy mechanics, platform traps |
|
| 440 |
+
| [`docs/REPRODUCIBILITY.md`](docs/REPRODUCIBILITY.md) | what a third party can and cannot reproduce |
|
| 441 |
+
| [`docs/RESEARCH_NOTES.md`](docs/RESEARCH_NOTES.md) | findings that changed the code; the router defect; negative results |
|
| 442 |
+
| [`docs/LIMITATIONS.md`](docs/LIMITATIONS.md) | the honest catalogue of everything not done or done poorly |
|
| 443 |
+
| [`docs/CHANGELOG.md`](docs/CHANGELOG.md) | versioned record of what changed and what was verified |
|
| 444 |
+
|
| 445 |
## Routing and the execution trace
|
| 446 |
|
| 447 |
Routing is deliberately two-stage, and the split matters:
|
|
|
|
| 453 |
decide the task actually dispatched. This is why a reading of `change` with **one** asset dispatches
|
| 454 |
`change_vqa` β the documented quantifier upgrade.
|
| 455 |
|
| 456 |
+
The server-side router has the same two-part shape in Python: `router/classifier.py::IntentRouter.route()`
|
| 457 |
+
produces an `Intent`, and `core/planner.py` turns that `Intent` into an ordered `ExecutionPlan`. The
|
| 458 |
+
planner's docstring states the rule it exists to enforce: **the planner is the only component permitted
|
| 459 |
+
to choose what runs; its input is a router prediction, its output is a plan, and the router's `Intent` is
|
| 460 |
+
an input to a decision, never the decision.** A design where `intent.task` selects a specialist in one
|
| 461 |
+
step has collapsed *understand* into *decide*, and three things break: the router can never be overruled,
|
| 462 |
+
a query needing two specialists can never get both, and there is no auditable record of the decision.
|
| 463 |
+
|
| 464 |
+
### The five router heads
|
| 465 |
+
|
| 466 |
+
The learned router is a small adapter over frozen MiniLM embeddings. `router/label_space.py` is the single
|
| 467 |
+
source of truth for its output ontology β both the dataset generator and the training script import from
|
| 468 |
+
it, so a class added in one place cannot silently desynchronise the other.
|
| 469 |
+
|
| 470 |
+
| Head | Classes / kind | Loss |
|
| 471 |
+
|---|---|---|
|
| 472 |
+
| `task` | 6 classes: `vqa, caption, grounding, change, optical_sar, unsupported` | softmax cross-entropy |
|
| 473 |
+
| `modality` | 4 classes: `optical, sar, optical_sar, unknown` | softmax cross-entropy |
|
| 474 |
+
| `temporal` | binary | single logit + `BCEWithLogitsLoss` |
|
| 475 |
+
| `spatial_output` | binary | single logit + `BCEWithLogitsLoss` |
|
| 476 |
+
| `language_output` | binary | single logit + `BCEWithLogitsLoss` |
|
| 477 |
+
|
| 478 |
+
The three binary heads use a single logit rather than a two-way softmax: a 2-way softmax would waste a
|
| 479 |
+
parameter and make the loss harder to weight. Because the heads are independent by construction, a
|
| 480 |
+
confident task label with an incoherent binary head is possible; the classifier resolves that in favour
|
| 481 |
+
of the task label (the controller keys off the task), and records that it did so.
|
| 482 |
+
|
| 483 |
+
Router configuration (`configs/base.yaml` Β§`router`):
|
| 484 |
+
|
| 485 |
+
| Key | Value |
|
| 486 |
+
|---|---|
|
| 487 |
+
| `model` | `sentence-transformers/all-MiniLM-L6-v2` |
|
| 488 |
+
| `revision` | `1110a243fdf4` |
|
| 489 |
+
| `max_length` | 128 |
|
| 490 |
+
| `embedding_dim` | 384 |
|
| 491 |
+
| `hidden_dim` | 128 |
|
| 492 |
+
| `dropout` | 0.10 |
|
| 493 |
+
| `confidence_threshold` | 0.70 |
|
| 494 |
+
| `num_tasks` | 6 |
|
| 495 |
+
| `training.epochs` / `batch_size` / `learning_rate` | 60 / 64 / 0.001 |
|
| 496 |
+
| `training.weight_decay` | 0.01 |
|
| 497 |
+
| `training.task_loss_weight` / `modality_loss_weight` / `binary_loss_weight` | 1.0 / 0.3 / 0.5 |
|
| 498 |
+
| `training.val_ratio` | 0.15 |
|
| 499 |
+
| `training.hard_negatives_to_test` | true |
|
| 500 |
+
|
| 501 |
+
**Finding F4-1 β the tokenizer ceiling.** The MiniLM tokenizer's own ceiling is **256** (verified by
|
| 502 |
+
probe). The project truncates to **128** β a deliberate truncation *well inside* the ceiling, not the
|
| 503 |
+
model limit. Satellite queries are short; halving the sequence halves attention cost for no measurable
|
| 504 |
+
accuracy loss. The encoder **asserts** `max_length β€ 256`, because truncating above the ceiling is a
|
| 505 |
+
silent no-op.
|
| 506 |
+
|
| 507 |
+
**Finding F4-2 β the router needs no GPU.** The encoder is frozen, so embeddings are cached and the
|
| 508 |
+
50,822-parameter adapter trains on cached vectors. **Measured on CPU: 20 epochs / 4,096 vectors in
|
| 509 |
+
0.28 s.**
|
| 510 |
+
|
| 511 |
+
**Finding F4-3 β splits must be by group.** Splits are by **group** (template / hard-negative family),
|
| 512 |
+
never by example. Hard-negative families are placed in the **test** split so their accuracy measures
|
| 513 |
+
generalisation rather than memorisation; splitting by example would leak template variants across the
|
| 514 |
+
boundary.
|
| 515 |
+
|
| 516 |
+
### The confidence gate and the fallback
|
| 517 |
+
|
| 518 |
+
`IntentRouter.route()` runs the learned router first, and falls back to the lexical rules only when the
|
| 519 |
+
learned router is below the confidence gate (or when the encoder cannot be loaded at all):
|
| 520 |
+
|
| 521 |
+
```
|
| 522 |
+
query
|
| 523 |
+
β encoder.encode (frozen MiniLM, 384-d)
|
| 524 |
+
β adapter (5 heads, softmax / sigmoid)
|
| 525 |
+
β confidence gate (router.confidence_threshold = 0.70)
|
| 526 |
+
β lexical fallback (only if below the gate)
|
| 527 |
+
β Intent (validated pydantic model)
|
| 528 |
+
```
|
| 529 |
+
|
| 530 |
+
The router **never refuses to answer**; `above_threshold` carries the uncertainty, and the planner β not
|
| 531 |
+
the router β decides what to do about it. The fallback is purely lexical: no model, no embeddings, no
|
| 532 |
+
randomness, ordered rules with the highest specificity first, and it **never invents capability** β if
|
| 533 |
+
nothing matches it returns `unsupported` with low confidence rather than guessing a task. Its precedence
|
| 534 |
+
is explicit, and it matters because the phrasings overlap:
|
| 535 |
+
|
| 536 |
+
```
|
| 537 |
+
dual_modality > temporal > spatial > caption > vqa > unsupported
|
| 538 |
+
```
|
| 539 |
+
|
| 540 |
+
Two examples of why precedence is load-bearing, both from the source:
|
| 541 |
+
|
| 542 |
+
- *"show me where the change happened"* has both a spatial term and a temporal term β `change` +
|
| 543 |
+
`spatial_output=True`.
|
| 544 |
+
- *"compare optical and radar to locate built-up areas"* has dual-modality **and** spatial β `optical_sar`
|
| 545 |
+
(spatial does not apply to the joint workflow, whose output is a classification).
|
| 546 |
+
|
| 547 |
+
The fallback's confidence band (0.72β0.92) **overlaps and can exceed** the trained model's β on the spec's
|
| 548 |
+
own examples the fallback returns 0.850β0.920 against the trained model's 0.780β1.000. So the planner
|
| 549 |
+
applies a **provenance discount** to its own reading of the confidence and never edits
|
| 550 |
+
`Intent.confidence` itself: a lexical fallback at 0.9 is not the same evidence as a learned model at 0.9,
|
| 551 |
+
and treating them identically would let a matched keyword outrank the model it fell back from.
|
| 552 |
+
`IntentRouter.adapter_source` returns `'trained'` or `'lexical_fallback'` so a caller can check this in
|
| 553 |
+
one place instead of inferring it from a confidence band β because `from_config` defaults `adapter_path`
|
| 554 |
+
to `None`, meaning the default router runs the **fallback**, not the trained adapter.
|
| 555 |
+
|
| 556 |
+
### A router bug worth recording
|
| 557 |
+
|
| 558 |
+
An earlier revision evaluated the temporal rule before the location rule, so *"Where are the built-up
|
| 559 |
+
areas in this image?"* matched `\bbuilt\b` as a *change* marker and `area` inside *"areas"* as a
|
| 560 |
+
quantifier. With one asset it collapsed to `vqa` and answered **"River"**. Fixed on 2026-09-25 in
|
| 561 |
+
`frontend/assets/js/mission.js`; the fix is covered by regression tests and verified live. The same defect
|
| 562 |
+
existed on a second surface (`SQ.policy` in `core.js`) and was fixed the same day.
|
| 563 |
+
|
| 564 |
+
The fix is four lexical changes, each documented in the source because each was a real failure:
|
| 565 |
+
|
| 566 |
+
| Change | Why |
|
| 567 |
+
|---|---|
|
| 568 |
+
| `built` removed from the temporal term set entirely | *"built-up areas"* is land-cover vocabulary, not a change marker. While it sat in the temporal set, the location question *"Where are the built-up areas in this image?"* was read as a change request and answered with the degenerate one-word *"River"*. Measured live, 2026-09-25. |
|
| 569 |
+
| `\barea\b` instead of bare `area` | Without the boundary the substring matched inside *"areas"*, so the already-mis-read change question was upgraded **again** to `change_vqa`. The boundary keeps the quantifier reading for a real *"how much area changed"* while refusing the plural land-cover noun. |
|
| 570 |
+
| `new` counts as a change marker **only** when the query is not a `where` question | The repo ships `eo/new-airport.jpg`, so *"Where is the new airport?"* is a real question, and `new` is a place descriptor as often as a change marker. |
|
| 571 |
+
| the change stem is matched **without** a trailing `\b` | `\bchang\b` cannot match *"changed"*, *"changes"* or *"changing"* β there is no word boundary between the stem and its inflection. With the boundary, the page's own default question (*"What changed here?"*) fell through to the `vqa` branch, so the change path was unreachable from the UI that exists to reach it. |
|
| 572 |
+
|
| 573 |
+
Both defect queries now dispatch to `grounding` and are captured in the screenshot set below.
|
| 574 |
|
| 575 |
### The eight execution events
|
| 576 |
|
|
|
|
| 589 |
| 8 | `RESULT_ASSEMBLED` | The `ResultEnvelope` is complete |
|
| 590 |
|
| 591 |
These are a **frontend** vocabulary driven by observable events β not a backend protocol and not a
|
| 592 |
+
model's reasoning trace. The run engine is deliberately dumb: it renders whatever events it receives, and
|
| 593 |
+
swapping the mock driver for a websocket/SSE feed of the same event names is the entire integration
|
| 594 |
+
surface. On live runs the trace bar reaches **94.4444 %** (17/18) and every node is marked live; the
|
| 595 |
+
preview path is the only source of mock-marked nodes.
|
| 596 |
+
|
| 597 |
+
The trace is deliberately *not* chain-of-thought. `ExecutionTrace` (`core/schemas.py`) carries
|
| 598 |
+
`run_id`, `schema_version`, `task`, `query`, `inputs`, `modalities`, `intent`, `validation`, `workflow`,
|
| 599 |
+
`steps`, `selected_models`, `parameters`, `outputs`, `confidence`, `timings`, `fallbacks`, `errors`,
|
| 600 |
+
`contradiction`, `config_hash`, `started_at`, `finished_at` β observable facts, with no field for model
|
| 601 |
+
reasoning and no LLM-generated confidence. The router's own trace projection is the model of this: it
|
| 602 |
+
emits the task, modality, the three booleans, the rounded confidence, the source, `above_threshold`,
|
| 603 |
+
`used_fallback` and `fallback_rule` β and nothing else.
|
| 604 |
|
| 605 |
## Real inference vs. the preview path
|
| 606 |
|
|
|
|
| 614 |
The distinction is observable, not asserted: a live run shows `live Β· N evidence Β· transport β¦` and
|
| 615 |
zero `.trace__node.is-mock` elements.
|
| 616 |
|
| 617 |
+
Two related traps are worth stating because they are the kind of thing a reader will otherwise
|
| 618 |
+
misdiagnose:
|
| 619 |
+
|
| 620 |
+
- **A pair-requiring task with one asset is refused, not silently downgraded server-side.** With one
|
| 621 |
+
asset, `change` answers `invalid_request` (*"change requires exactly 2 assets (T1 and T2); got 1"*) and
|
| 622 |
+
the whole envelope comes back `degraded: true`. Measured live, 2026-09-25. The console's job is to
|
| 623 |
+
avoid asking for a pair-requiring task when only one asset exists; it does so in `chooseTask()`, and it
|
| 624 |
+
**names the substitution** rather than hiding it.
|
| 625 |
+
- **The asset set sent is per-task.** `vqa`, `grounding` and `caption` accept a single image, while
|
| 626 |
+
`change`, `change_vqa` and `optical_sar` require a pair. Sending the optional earlier frame to a
|
| 627 |
+
single-image task makes the backend reject the whole request β measured live when a pair was uploaded
|
| 628 |
+
and a VQA question asked. The fix isolates the file set so the pair is only ever sent to the tasks that
|
| 629 |
+
declared it.
|
| 630 |
+
|
| 631 |
## Models
|
| 632 |
|
| 633 |
Six trained artifacts are released. **Four are task heads and two are adapters** β none is a complete
|
|
|
|
| 646 |
Backbones are third-party and pinned by `repo_id` + `revision` in `configs/base.yaml`; they are
|
| 647 |
fetched from the Hugging Face Hub, not redistributed here.
|
| 648 |
|
| 649 |
+
### The six artifacts, byte-for-byte
|
| 650 |
+
|
| 651 |
+
`models/manifest.json` is **generated by reading the files** β no byte count or hash is typed by hand. Its
|
| 652 |
+
schema is `satquery_model_manifest_v1`, generated 2026-09-25T18:15:38+00:00, `artifact_count: 6`, and
|
| 653 |
+
every entry carries the frozen `config_hash` `78f1e3700da15aa1`.
|
| 654 |
+
|
| 655 |
+
| # | `id` | Task | Kind | Local path | HF path | Bytes | sha256 (first 16) |
|
| 656 |
+
|---|---|---|---|---|---|---|---|
|
| 657 |
+
| 1 | `change_head` | `change` | trained head | `artifacts/change/levir_change_v001/head.pt` | `change/head.pt` | 63,231,009 | `c5ef31277b67aa01` |
|
| 658 |
+
| 2 | `change_vqa_head` | `change_vqa` | trained head | `artifacts/change_vqa/run/head.pt` | `change_vqa/head.pt` | 5,822,809 | `cfae5e43b97ca930` |
|
| 659 |
+
| 3 | `optical_sar_fusion_head` | `optical_sar` | trained head | `artifacts/optical_sar/fusion_head_production_v001/head.pt` | `optical_sar/head.pt` | 14,427,457 | `785815729a3a39fc` |
|
| 660 |
+
| 4 | `grounding_head` | `grounding` | trained head | `artifacts/grounding/remoteclip_grounding_v001/head.pt` | `grounding/head.pt` | 12,639,041 | `93432f7034be91a8` |
|
| 661 |
+
| 5 | `router_adapter` | `router` | trained adapter | `artifacts/router/router_adapter_v001/adapter.pt` | `router/adapter.pt` | 211,961 | `8527c3ed28a293e1` |
|
| 662 |
+
| 6 | `vlm_lora_adapter` | `vlm` | LoRA adapter | `.scratch/phase6_real_adapter/phase6_adapter/adapter_model.safetensors` | `vlm/adapter_model.safetensors` | 34,798,048 | `07c76a75fa046248` |
|
| 663 |
+
|
| 664 |
+
Total released weight payload: **131,130,325 bytes (~125 MiB)**. Two of the six hashes are cross-checked
|
| 665 |
+
against values recorded **independently** elsewhere in the project β `change_vqa_head` against
|
| 666 |
+
`artifacts/change_vqa/run/PROMOTION.json`, and `vlm_lora_adapter` against the adapter's own provenance
|
| 667 |
+
manifest β and both agree. That is an external cross-check, not a self-consistency claim.
|
| 668 |
+
|
| 669 |
+
The manifest also records each artifact's architecture and the metric artifact it came from:
|
| 670 |
+
|
| 671 |
+
| `id` | Architecture | Source metric artifact |
|
| 672 |
+
|---|---|---|
|
| 673 |
+
| `change_head` | STANet-style Siamese change detector (ResNet-18 + PAM) | `artifacts/change/eval_test/eval_result.json` |
|
| 674 |
+
| `change_vqa_head` | `change_vqa_head_v1` (1,453,912 parameters) | `artifacts/change_vqa/run/PROMOTION.json` |
|
| 675 |
+
| `optical_sar_fusion_head` | CROMA-base fusion head (input 2318 β hidden 512 β 19 classes) | `artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json` |
|
| 676 |
+
| `grounding_head` | RemoteCLIP ViT-B/32 grounding head (feature 2048, hidden 512) | `artifacts/grounding/remoteclip_grounding_v001/eval_result_canonical.json` |
|
| 677 |
+
| `router_adapter` | task/modality adapter over frozen MiniLM embeddings (~50,822 params) | `artifacts/router/threshold_sweep_val.json` |
|
| 678 |
+
| `vlm_lora_adapter` | PEFT LoRA (r=16, Ξ±=32, dropout 0.05) on text-model projections | `artifacts/vlm/phase6_closure.json` |
|
| 679 |
+
|
| 680 |
+
Training checkpoints also exist (`checkpoint_last.pt` at 189,291,829 B for change; `checkpoint_last.pt` at
|
| 681 |
+
12,640,331 B for grounding; `checkpoint-1500` / `checkpoint-2000` for the LoRA adapter) and are
|
| 682 |
+
**not** the released artifacts β they are archived as provenance.
|
| 683 |
+
|
| 684 |
+
### The Hugging Face release
|
| 685 |
+
|
| 686 |
+
The six artifacts are published at **https://huggingface.co/thundercode/SatQuery** (public,
|
| 687 |
+
`private: false`, `gated: false`), HEAD `55681e0cddb91a4a5655da98a49bc025e537b657`, 22 files, last
|
| 688 |
+
modified `2026-09-25T18:21:52Z`.
|
| 689 |
+
|
| 690 |
+
The release was verified by **re-downloading each artifact over direct HTTPS and hashing the bytes
|
| 691 |
+
received**, rather than trusting the upload step: 6/6 `MATCH`, 0 failed, and the four support files
|
| 692 |
+
(`README.md`, `MODEL_CARD.md`, `models/manifest.json`, `models/checksums.sha256`) confirmed present. The
|
| 693 |
+
pre-existing content was a 25-byte stub README (literally `---\nlicense: unknown\n---`) which was
|
| 694 |
+
replaced, and the standard HF LFS routing `.gitattributes`, which was left untouched.
|
| 695 |
+
|
| 696 |
+
> **A verification method that was itself wrong (recorded).** The *first* verification attempt reported
|
| 697 |
+
> all six artifacts `DIFFER`, with every remote hash equal to
|
| 698 |
+
> `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855` β the sha256 of **empty content**.
|
| 699 |
+
> The cause was not the upload: `hf_hub_download` returned an empty file in this environment, so the
|
| 700 |
+
> verifier hashed nothing. It was caught by a second, independent method (a direct `curl` download), which
|
| 701 |
+
> produced the correct hash for `router/adapter.pt` and confirmed it to be a real PyTorch zip (`PK\x03\x04`).
|
| 702 |
+
> The verifier was then rewritten to use direct HTTPS with proxies disabled. The failed first attempt is
|
| 703 |
+
> recorded because a verifier that silently hashes an empty file would have produced a **false failure** β
|
| 704 |
+
> and, with a different bug, could just as easily have produced a **false pass**.
|
| 705 |
+
|
| 706 |
+
**No secret was uploaded.** The uploaded set is the model card, the manifest, the checksums, the docs, and
|
| 707 |
+
the six weight files; no tokens, keys, environment files or credentials exist in any uploaded file, and
|
| 708 |
+
the token used for the upload is not written into any released file.
|
| 709 |
+
|
| 710 |
## Measured results
|
| 711 |
|
| 712 |
Every number below traces to an artifact, a test, or a live run. **Nothing here is a system-level
|
|
|
|
| 734 |
| Router **overall ungated** accuracy | 0.965116 | val, n=86, corpus-limited | `overall_ungated_accuracy` | **TEST NOT RUN** |
|
| 735 |
| System-level end-to-end benchmark | β | β | β | **NOT RUN β none exists** |
|
| 736 |
|
| 737 |
+
### Source artifacts and the rules the table follows
|
| 738 |
+
|
| 739 |
+
Each metric family has exactly one source artifact:
|
| 740 |
+
|
| 741 |
+
| Metric family | Artifact |
|
| 742 |
+
|---|---|
|
| 743 |
+
| change | `artifacts/change/eval_test/eval_result.json` |
|
| 744 |
+
| grounding | `artifacts/grounding/remoteclip_grounding_v001/eval_result_canonical.json`, `β¦_matched6.json` |
|
| 745 |
+
| optical-SAR | `artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json` |
|
| 746 |
+
| change-VQA | `artifacts/change_vqa/run/PROMOTION.json` |
|
| 747 |
+
| router | `artifacts/router/threshold_sweep_val.json` |
|
| 748 |
+
| calibration | `artifacts/calibration_v001.json` |
|
| 749 |
+
| VLM | `artifacts/vlm/phase6_closure.json` |
|
| 750 |
+
|
| 751 |
+
All **20** quoted metrics are checked against these files by
|
| 752 |
+
[`release/tools/verify_readme_metrics.py`](https://github.com/Anish-lab-blip/SatQuery-AI), which resolves
|
| 753 |
+
nested artifact keys (including keys that themselves contain dots β the `recall` dict is keyed
|
| 754 |
+
`"0.10"/"0.25"/"0.50"`, so a naive `split(".")` walk would break) and compares each value at the precision
|
| 755 |
+
printed here. It exits non-zero if any claim fails and prints `ALL CLAIMS VERIFIED` only when everything
|
| 756 |
+
matches. The table obeys six rules:
|
| 757 |
+
|
| 758 |
+
1. **Two protocols are never collapsed.** Grounding is reported under *both* the canonical and matched6
|
| 759 |
+
protocols. Quoting 0.2838 alone would be selective.
|
| 760 |
+
2. **Two test sets are never collapsed.** Change-VQA is reported on `test` **and** `test2`.
|
| 761 |
+
3. **accuracy never travels without macro-F1.** For imbalanced multi-class heads (optical-SAR,
|
| 762 |
+
change-VQA) the macro-F1 is reported alongside accuracy, always.
|
| 763 |
+
4. **Validation is not test.** The router number is labelled "overall **ungated** accuracy", val, n = 86.
|
| 764 |
+
5. **A negative result stays negative.** Calibration ECE worsened and is shown worsening.
|
| 765 |
+
6. **USABLE β ACCEPTED.** The VLM metrics are real; the artifact is nevertheless acceptance-rejected.
|
| 766 |
+
|
| 767 |
+
There is also **no composite or vanity score**: no single headline accuracy for the system, and none
|
| 768 |
+
invented by averaging the per-task numbers.
|
| 769 |
+
|
| 770 |
### Grounding: three decode variants, two protocols
|
| 771 |
|
| 772 |
+
The grounding head is evaluated under **two matching protocols** (canonical, matched6) and **three**
|
| 773 |
+
decode variants. Quoting a single number would misrepresent the result, so all of them are listed:
|
| 774 |
|
| 775 |
| Decode | canonical mean_best_IoU | matched6 mean_best_IoU |
|
| 776 |
|---|---|---|
|
|
|
|
| 782 |
argmax decode (0.1215) is barely better than zero-shot. The absolute level is modest either way:
|
| 783 |
**grounding is useful, not solved.**
|
| 784 |
|
| 785 |
+
The recall@0.5 numbers travel with the IoU numbers: canonical **0.2198**, matched6 **0.1938**. The box
|
| 786 |
+
convention is a common source of silent error, which is why the project converts VRSBench's 0β100 boxes
|
| 787 |
+
to its own 0β1 convention through a *declared* `benchmark_box_scale: 100.0` β so the conversion cannot be
|
| 788 |
+
applied twice or forgotten β and reports both protocols.
|
| 789 |
+
|
| 790 |
+
The head itself is a trainable head over the frozen RemoteCLIP ViT-B/32 encoder. Its per-cell feature is
|
| 791 |
+
`concat([patch, text, patchΒ·text, global_pool])` = `4 Γ 512 = 2048` (finding P7-1: the transformer width
|
| 792 |
+
is 768, but `visual.proj` maps to a projected dim of **512**). Cells are assigned by ground-truth box
|
| 793 |
+
centre (`cell_relative` decode). The objectness BCE is weighted **20Γ** because only ~1 of 49 cells is
|
| 794 |
+
positive; unweighted, the optimum collapses to "no object" everywhere. Image resolution is frozen at
|
| 795 |
+
**224** β 448 was evaluated and **rejected** (see below).
|
| 796 |
+
|
| 797 |
+
### The grounding resolution decision β a pre-registered rejection
|
| 798 |
+
|
| 799 |
+
**Question:** should grounding decode at 448 or 224? **Answer: 224. 448 was rejected** β and the rejection
|
| 800 |
+
is notable because it was *pre-registered* and then *confirmed* by a paired test over identical samples
|
| 801 |
+
(n = 16,159):
|
| 802 |
+
|
| 803 |
+
| Comparison (448 vs 224) | Value |
|
| 804 |
+
|---|---|
|
| 805 |
+
| mean best IoU | **β0.0147** |
|
| 806 |
+
| recall@0.5 | β0.0022 |
|
| 807 |
+
| recall@0.10 | β0.0699 |
|
| 808 |
+
| recall@0.25 | β0.0243 |
|
| 809 |
+
| latency | **1.59Γ** |
|
| 810 |
+
| paired 95 % CI | [β0.0160, β0.0134] |
|
| 811 |
+
| paired t | **β22.63** |
|
| 812 |
+
| 448 better on | 8.5 % of records |
|
| 813 |
+
| 448 worse on | **20.9 %** of records |
|
| 814 |
+
|
| 815 |
+
448 lost on **every** axis. The pre-registered decision rule and the paired test **agree** on 224. This is
|
| 816 |
+
a model of how a resolution decision should be made: declared in advance, then tested. Recorded as
|
| 817 |
+
`RESOLVED 2026-09-16` in `configs/base.yaml` and in
|
| 818 |
+
[`docs/RESEARCH_NOTES.md`](docs/RESEARCH_NOTES.md) Β§2.
|
| 819 |
+
|
| 820 |
### Fusion: measured but the ruling is open
|
| 821 |
|
| 822 |
Optical-SAR fusion reaches **0.931 accuracy** on a 19-class held-out set of 4,000 β but only
|
|
|
|
| 824 |
frequent classes and weak on rare ones. The acceptance ruling for this head is **OPEN**, and the
|
| 825 |
headline accuracy must never be quoted without the macro_F1 beside it.
|
| 826 |
|
| 827 |
+
The metric JSON records why the macro score is low by construction: of the 19 classes in the label space,
|
| 828 |
+
**14 are present** and **5 are absent** in the scored split, and the **macro-F1 denominator is all 19**
|
| 829 |
+
(absent classes contribute 0.0). It also records `is_deciding_statistic: False` β this is a reported
|
| 830 |
+
measurement, not a decision statistic. The feature concatenation is the frozen one:
|
| 831 |
+
|
| 832 |
+
```
|
| 833 |
+
input_dim = 3 Γ 768 + 12 + 2 = 2318 β hidden 512 β num_classes 19 (BigEarthNet CLC)
|
| 834 |
+
```
|
| 835 |
+
|
| 836 |
+
and the **availability mask is consumed by the fusion head, not by CROMA** (finding C-1) β CROMA always
|
| 837 |
+
sees the canonical channel counts (12 optical, 2 SAR).
|
| 838 |
+
|
| 839 |
+
Two further caveats on this head, stated rather than hidden:
|
| 840 |
+
|
| 841 |
+
- **The live service returns a bare class index** (`class_18`), not a human-readable label. The modality
|
| 842 |
+
accounting in the response confirms the right channels reached the fusion head, but the presentation is
|
| 843 |
+
not user-facing.
|
| 844 |
+
- **The local BigEarthNet subset is 100 % single-label**, against the official 1β11 multi-label scheme, so
|
| 845 |
+
its metrics are **not comparable** to published BigEarthNet numbers. Any statement of the form
|
| 846 |
+
"BigEarthNet mAP = X" is false for this subset.
|
| 847 |
+
|
| 848 |
### Change-VQA: two test sets, and they disagree
|
| 849 |
|
| 850 |
`artifacts/change_vqa/run/PROMOTION.json` records **two** test evaluations:
|
|
|
|
| 857 |
The `test` numbers are the higher pair. Both are reported here; quoting only `test` would overstate
|
| 858 |
the result. The acceptance ruling is **OPEN**.
|
| 859 |
|
| 860 |
+
The head was trained **outside this repository**, on an external GPU (Kaggle), and promoted through a
|
| 861 |
+
byte-identity gate: sha256 `cfae5e43b97ca930f568dc5b8ae4f36b24e9ff717af226159802206ffd63a82a`, 5,822,809
|
| 862 |
+
bytes, architecture `change_vqa_head_v1`, 1,453,912 parameters, **0 non-finite tensors**, weights not
|
| 863 |
+
modified during promotion, byte-identical to source, hash agreeing across `model_metadata.json`,
|
| 864 |
+
`run_record.json` and `hashes.json`. Selection was epoch **8**, chosen on **val answer accuracy
|
| 865 |
+
0.700018**, stopped by early stopping; seed 42. It trains on **cached change + text features**, not on
|
| 866 |
+
raw imagery β the raw CDVQA loader loads examples but has no training loop of its own, and the two paths
|
| 867 |
+
are not conflated.
|
| 868 |
+
|
| 869 |
### Phase 6 / VLM: deployment success β model acceptance
|
| 870 |
|
| 871 |
The Phase-6 SmolVLM LoRA adapter reaches **exact_match 0.963** and **F1 0.96432** on a frozen
|
|
|
|
| 874 |
Those two verdicts are not in conflict, and the distinction is the point:
|
| 875 |
|
| 876 |
- **USABLE_VERIFIED** β the adapter loads, runs, and produces the measured numbers in the deployed
|
| 877 |
+
pipeline. The F1 is **+49.5 pp** over the unadapted baseline.
|
| 878 |
+
- **ACCEPTANCE-REJECTED** β the change did not clear the project's own pre-registered acceptance bar. The
|
| 879 |
+
record of *why* is stored in `artifacts/vlm/phase6_closure.json` under `why_acceptance_rejected`; the
|
| 880 |
+
artifact's `status` is `CLOSED`.
|
| 881 |
|
| 882 |
A model can be a working engineering artifact and a rejected research result at the same time.
|
| 883 |
+
This release keeps both labels. The deployed caption/VQA path therefore uses the **unadapted** SmolVLM.
|
| 884 |
+
|
| 885 |
+
The adapter's own shape is recorded: PEFT **0.19.1**, `r = 16`, `alpha = 32`, `dropout = 0.05`, targeting
|
| 886 |
+
`model.text_model.*.{q,k,v,o,gate,up,down}_proj`, precision **fp16** (finding C-6: the T4 is compute
|
| 887 |
+
capability 7.5, so fp16 β **not** bf16), batch size 2, gradient accumulation 8, learning rate 0.0002,
|
| 888 |
+
1 epoch, gradient checkpointing on, `save_every_steps` 500.
|
| 889 |
|
| 890 |
### Calibration: it got worse, and we say so
|
| 891 |
|
| 892 |
The `change_vqa` confidence path applies temperature scaling (`T = 0.9773`). Measured on the
|
| 893 |
validation split (n=16441):
|
| 894 |
|
| 895 |
+
| | ECE | NLL |
|
| 896 |
+
|---|---|---|
|
| 897 |
+
| Before temperature scaling | **0.013755** | 0.689741 |
|
| 898 |
+
| After temperature scaling | **0.014929** | 0.689631 |
|
| 899 |
|
| 900 |
+
**Calibration did not improve β it moved slightly worse.** The fitted temperature is `0.9772732` and
|
| 901 |
+
`ece_improvement` is **β0.001174**: negative. The scaling is retained because it is part
|
| 902 |
of the frozen configuration, not because it helped. The reliability curve plotted on the Benchmark page
|
| 903 |
is explicitly labelled as the **pre-scaling** diagram so a reader cannot mistake it for the calibrated
|
| 904 |
+
result. The calibrated curve is **not plotted**.
|
| 905 |
+
|
| 906 |
+
### What is NOT benchmarked
|
| 907 |
+
|
| 908 |
+
| Benchmark | Status | Note |
|
| 909 |
+
|---|---|---|
|
| 910 |
+
| **System-level end-to-end accuracy** | **NOT RUN β none exists** | There is no measured end-to-end benchmark of the full router β specialist β envelope pipeline. No such number is claimed anywhere. |
|
| 911 |
+
| **Router test split** | **NOT RUN** | Only the validation split (n = 86) was scored. |
|
| 912 |
+
| **Benchmark adapters** | **NOT RUN** | Adapter-based benchmark runs were not executed. |
|
| 913 |
+
| **Efficiency / latency benchmark** | not systematically measured | Per-specialist latency is recorded incidentally in artifacts (e.g. grounding `latency_ms_per_image` 2.205 ms for the head), but there is no end-to-end latency benchmark. |
|
| 914 |
+
| **Cross-dataset generalisation** | **NOT RUN** | Each specialist is evaluated only on its own training-family test split. |
|
| 915 |
+
| **Human evaluation** | **NOT RUN** | β |
|
| 916 |
+
| **Adversarial / robustness evaluation** | **NOT RUN** | β |
|
| 917 |
|
| 918 |
## Live validation
|
| 919 |
|
| 920 |
Validation drove the **production site** in a headed browser, one upload per case, with per-case
|
| 921 |
+
screenshots and recorded run identifiers. It is **behavioural** evidence β that the pipeline runs and
|
| 922 |
+
routes correctly β and it is **not** an accuracy claim; accuracy and behaviour are evaluated separately.
|
| 923 |
|
| 924 |
| Property | Result |
|
| 925 |
|---|---|
|
|
|
|
| 932 |
| Trace fill | 94.4444 % on every live run |
|
| 933 |
| Frontend regression suite | **106 passed** (`tests/unit/test_frontend_live_wiring.py`) |
|
| 934 |
|
| 935 |
+
Each pass produced **fresh run identifiers** β no run id is shared between passes. The three passes ran
|
| 936 |
+
against two frontend revisions:
|
| 937 |
+
|
| 938 |
+
| Pass | Deployed HEAD | Result |
|
| 939 |
+
|---|---|---|
|
| 940 |
+
| 1 | `ff46eba42b18` + `d413d3672311` | 8/8 |
|
| 941 |
+
| 2 | `2d7ae53b482d` | 8/8 |
|
| 942 |
+
| 3 | `2d7ae53b482d` | 8/8 |
|
| 943 |
|
| 944 |
The harness asserts the form state **before** dispatch β that the query box really holds the intended
|
| 945 |
query, that `#obsTail` reads `ready`, and that both frames are attached for pair tasks. This matters:
|
| 946 |
an earlier harness revision typed with synthetic key events that Chrome silently drops when the window
|
| 947 |
lacks OS focus, so it dispatched the page's *default* query and still recorded a "result". The
|
| 948 |
+
assertions exist because of that failure. The earlier 8/8 run was independently re-examined and confirmed
|
| 949 |
+
**not** to have been infected (its answers were query-specific and the query text was embedded in the
|
| 950 |
+
answers), but the failure mode is recorded because it is exactly the kind of silent false-positive an
|
| 951 |
+
evaluation harness must never have.
|
| 952 |
+
|
| 953 |
+
Deployed-artifact integrity was checked separately: **9 files** were re-read from the GitHub API and
|
| 954 |
+
compared byte-for-byte against local copies, and all 9 were **sha256 byte-identical**; the deployed HEAD
|
| 955 |
+
was re-read from the API.
|
| 956 |
|
| 957 |
### Representative real run IDs
|
| 958 |
|
|
|
|
| 972 |
|
| 973 |
The last two are the router-defect queries. Both previously collapsed to `vqa` and answered "River".
|
| 974 |
|
| 975 |
+
Note the fifth row: *"Did the coastline advance between the two observations?"* is read as `change` and
|
| 976 |
+
**dispatches `change_vqa`** β the documented quantifier upgrade, because the page's Answer block promises
|
| 977 |
+
an answer and the server's `change` returns a spatial map with no language output. With one asset
|
| 978 |
+
attached, `change` would instead be refused outright (*"requires exactly 2 assets"*); the console avoids
|
| 979 |
+
asking for a pair-requiring task when only one asset exists.
|
| 980 |
+
|
| 981 |
### Screenshots
|
| 982 |
|
| 983 |
Eight captures from the post-fix live run (headed browser, 1384Γ855, one upload per case). Each
|
|
|
|
| 1015 |
`configs/base.yaml`. The frozen config hash is **`78f1e3700da15aa1`** β the loader refuses to run a
|
| 1016 |
config that violates the recorded invariants (for example `fusion.input_dim == 3*encoder_dim + 12 + 2`).
|
| 1017 |
|
| 1018 |
+
### The invariants the loader enforces
|
| 1019 |
+
|
| 1020 |
+
`core/config.py` validates the registry at load time and raises `ConfigError` β naming every violation β
|
| 1021 |
+
rather than letting a bad value reach runtime. The invariants are not documentation; they are checks:
|
| 1022 |
+
|
| 1023 |
+
| Invariant | Why it exists |
|
| 1024 |
+
|---|---|
|
| 1025 |
+
| `croma.image_resolution % 8 == 0` | CROMA asserts this (finding C-7); native 120 β 225 patches |
|
| 1026 |
+
| `training.precision β {fp16, bf16, fp32}` | the T4 is SM 7.5, so bf16 is unavailable (finding C-6) |
|
| 1027 |
+
| `deployment.torch_compile is not true` | ZeroGPU does not support `torch.compile` (finding C-8) |
|
| 1028 |
+
| `vlm.processor_longest_edge β€ image.tile_size` | the processor's default `longest_edge` is 2048, which upscales a 512 px tile 4Γ and then splits it into **17** sub-images β a ~17Γ overrun, not the 4Γ the plan estimated (finding F5-2). Tying the pin to `image.tile_size` makes it a *control*, so the processor cannot silently start upscaling again. |
|
| 1029 |
+
| `vlm.prompt_must_use_chat_template is true` | SmolVLM raises `ValueError` on prompts lacking one `<image>` token per image (finding F5-3) |
|
| 1030 |
+
| `fusion.input_dim == 3*encoder_dim + optical_channels + sar_channels` (= 2318) | CROMA emits optical/SAR/joint GAP vectors; the availability mask is consumed by the head (finding C-1) |
|
| 1031 |
+
| `croma.optical_channels == 12` and `croma.sar_channels == 2` | CROMA's `s2_channels` / `s1_channels` are fixed |
|
| 1032 |
+
| `grounding_head.feature_dim == 4 * grounding.encoder_projected_dim` (= 2048) | a mismatch is a **silent** shape error β torch raises only at the similarity step, after patch features are already cached (finding P7-1) |
|
| 1033 |
+
| `router.tasks` includes `unsupported` and `router.num_tasks == len(router.tasks)` | the ontology and its declared size cannot drift apart |
|
| 1034 |
+
| `change.sa_mode β {BAM, PAM}` and `change.encoder` is set | the change architecture is not implicit |
|
| 1035 |
+
| `image.top_k_tiles β€ image.max_tiles` | the dispatch ceiling cannot exceed the examination ceiling |
|
| 1036 |
+
|
| 1037 |
+
Because a config edit moves `Config.hash` and invalidates every artifact keyed to it, deployment state
|
| 1038 |
+
that must not move the hash (asset-store capacity, TTL, the per-file cap) is read from the **environment**
|
| 1039 |
+
rather than from `configs/base.yaml` β the same reasoning that keeps the config hash frozen.
|
| 1040 |
+
|
| 1041 |
## Local development
|
| 1042 |
|
| 1043 |
```bash
|
|
|
|
| 1060 |
python -m pytest tests/unit/test_frontend_live_wiring.py -q
|
| 1061 |
```
|
| 1062 |
|
| 1063 |
+
### The test suites
|
| 1064 |
+
|
| 1065 |
+
| Suite | Command | Expected |
|
| 1066 |
+
|---|---|---|
|
| 1067 |
+
| Frontend live-wiring | `pytest tests/unit/test_frontend_live_wiring.py` | **106 passed** |
|
| 1068 |
+
| Doc/frontend suite | `pytest` on the 5 doc/frontend files | **183 passed** |
|
| 1069 |
+
| Full unit suite | `pytest tests/unit` | 5β6 **environmental** failures (sandbox delete guard Γ 4, 1 ordering flake, 1 stale adapter test) |
|
| 1070 |
+
|
| 1071 |
+
The full-suite failures are **not hidden**, and they are not regressions: 4 are the sandbox's bulk-delete
|
| 1072 |
+
guard (`test_safe_delete_shim`), 1 is an ordering flake that passes in isolation, and 1 is a stale adapter
|
| 1073 |
+
test (CROMA is now shipped). Re-running the affected files together gives **137 passed**, confirming the
|
| 1074 |
+
failures are attributable to the sandbox environment and test ordering rather than the code under test.
|
| 1075 |
+
|
| 1076 |
+
### Reproduce a live run
|
| 1077 |
+
|
| 1078 |
+
The deployed stack is reachable. Note the authoring sandbox has a dead proxy, so outbound calls need
|
| 1079 |
+
`--noproxy '*'` (curl) or `ProxyHandler({})` (Python):
|
| 1080 |
+
|
| 1081 |
+
```bash
|
| 1082 |
+
curl --noproxy '*' https://satquery-backend-m4yv.onrender.com/api/health
|
| 1083 |
+
curl --noproxy '*' https://satquery-backend-m4yv.onrender.com/api/capabilities
|
| 1084 |
+
```
|
| 1085 |
+
|
| 1086 |
+
`/api/capabilities` returns six tasks, all `available: true`. A live run requires the tunnel agent to be
|
| 1087 |
+
connected (`agent_connected: true`); if the Codespace is stopped, the request parks until the tunnel
|
| 1088 |
+
timeout. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) Β§5β6.
|
| 1089 |
+
|
| 1090 |
## Deployment
|
| 1091 |
|
| 1092 |
The live topology is **Cloudflare Pages β Render β outbound tunnel β GitHub Codespace**.
|
|
|
|
| 1102 |
Render β start the Codespace if stopped β poll `/v1/health` β surface *"Waking inference engineβ¦"* β
|
| 1103 |
`POST /infer` β result.
|
| 1104 |
|
| 1105 |
+
### Live revisions at this release
|
| 1106 |
+
|
| 1107 |
+
| Component | Repository | Visibility | Revision | Host |
|
| 1108 |
+
|---|---|---|---|---|
|
| 1109 |
+
| Frontend | `Anish-lab-blip/SatQuery-Frontend` | private | **`2d7ae53b482d`** | Cloudflare Pages β `satquery.pages.dev` |
|
| 1110 |
+
| Backend / orchestrator | `Anish-lab-blip/SatQuery-Backend` | private | **`89d80eaddec5`** | Render β `satquery-backend-m4yv.onrender.com` |
|
| 1111 |
+
| Inference | `Anish-lab-blip/SatQuery-Inference` | private | **`5a0936ace491`** | Codespace, port 8000, via outbound tunnel |
|
| 1112 |
+
| Public umbrella | `Anish-lab-blip/SatQuery-AI` | **public** | `3dcabd32da41` | this release home |
|
| 1113 |
+
|
| 1114 |
+
> **Trap.** `deploy/` inside the monorepo is **stale and untracked**. It is **not** the deployed source.
|
| 1115 |
+
> Edits must go to the three real repositories. Recorded in
|
| 1116 |
+
> [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) Β§1.
|
| 1117 |
+
|
| 1118 |
+
### Environment variables
|
| 1119 |
+
|
| 1120 |
+
Render (gateway), measured live:
|
| 1121 |
+
|
| 1122 |
+
| Variable | Value (live) |
|
| 1123 |
+
|---|---|
|
| 1124 |
+
| `CODESPACE_NAME` | `potential-space-trout-r4ppw969w45j2pvvw` |
|
| 1125 |
+
| `CODESPACE_PORT` | `8000` |
|
| 1126 |
+
| `SATQUERY_ALLOWED_ORIGINS` | `https://satquery.pages.dev` |
|
| 1127 |
+
| `SATQUERY_DEVICE` | `cpu` |
|
| 1128 |
+
| `SATQUERY_TRANSPORT` | `auto` |
|
| 1129 |
+
| `SATQUERY_TUNNEL_TIMEOUT_S` | `150` |
|
| 1130 |
+
| `SATQUERY_WAKE_TIMEOUT_S` | `120` |
|
| 1131 |
+
| `SATQUERY_UPSTREAM_TIMEOUT_S` | `90` |
|
| 1132 |
+
| `GITHUB_TOKEN` | present |
|
| 1133 |
+
|
| 1134 |
+
Codespace (inference):
|
| 1135 |
+
|
| 1136 |
+
| Variable | Purpose |
|
| 1137 |
+
|---|---|
|
| 1138 |
+
| `PORT` | platform-assigned; **must be read** |
|
| 1139 |
+
| `SATQUERY_DEVICE` | `cpu` \| `cuda` \| `mps` \| `null`; read **without importing torch** |
|
| 1140 |
+
| `SATQUERY_MAX_FILE_BYTES` | per-file cap (shared with Render) |
|
| 1141 |
+
| `SATQUERY_ASSET_ENABLED` / `SATQUERY_ASSET_DIR` | both required for `/v1/assets`; fails closed otherwise |
|
| 1142 |
+
| `SATQUERY_ASSET_MAX_FILES` / `SATQUERY_ASSET_TTL_S` | optional handle capacity / lifetime |
|
| 1143 |
+
|
| 1144 |
+
Deploy mechanics: the frontend is staged by `scripts/stage_pages.mjs` and deployed with
|
| 1145 |
+
`npx wrangler pages deploy`; the backend is a `render.yaml` blueprint whose `main.py` exposes `app`; the
|
| 1146 |
+
inference host runs `deploy/codespace/serve.py` on `$PORT` and the devcontainer forwards port 8000 and
|
| 1147 |
+
starts the tunnel agent on `postStartCommand`. Repository writes are performed through the GitHub Git
|
| 1148 |
+
Data API (blob β tree β commit β `PATCH` ref) with **sha256 byte-verification** of every uploaded blob,
|
| 1149 |
+
rather than `git push`, so each deployed file is verified by content hash.
|
| 1150 |
+
|
| 1151 |
### Deployment caveats
|
| 1152 |
|
| 1153 |
- **Cold start.** The inference host may be stopped when idle. The first request after a cold start
|
| 1154 |
can exceed the client timeout while weights are fetched; a retry a few seconds later normally
|
| 1155 |
succeeds. Warm the stack before any demonstration and confirm
|
| 1156 |
+
`GET /api/health` reports `tunnel.agent_connected: true`. Cold start is **tens of seconds** and is
|
| 1157 |
+
documented rather than papered over.
|
| 1158 |
- **Tunnel gaps.** The tunnel agent can be briefly absent. A request issued during such a gap may hang
|
| 1159 |
or return HTTP 504. **This is not fixed in production** β a prepared patch
|
| 1160 |
(`forward_unavailable` 503 / `upstream_timeout` 504 plus a `codespace_name` fix) exists and is
|
| 1161 |
documented, but it was deliberately not deployed. Root cause: in `auto` transport mode a tunnel
|
| 1162 |
+
timeout falls through to the forwarded-port path (`main.py:546`), which then spends the 120 s wake
|
| 1163 |
+
timeout on an HTTP 302 β the observed ~249 s failure (150 + 120).
|
| 1164 |
- **`codespace_name`** is still reported with a trailing newline by `/api/health` (cosmetic; the wake
|
| 1165 |
path strips it).
|
| 1166 |
+
- **Never retry `POST /api/infer` at the gateway** β a retry consumes inference twice.
|
| 1167 |
+
- **Platform traps, recorded so they are not rediscovered.** Cloudflare `_headers` rules **concatenate**
|
| 1168 |
+
rather than override, and Chromium takes the first `max-age` it encounters, so a later rule cannot "fix"
|
| 1169 |
+
an earlier one. Cloudflare 308-redirects `X.html` β `/X`, so the extensionless path must be referenced.
|
| 1170 |
+
A forwarded Codespace port returns 302 for a private repo β which is *why* the tunnel exists. And the
|
| 1171 |
+
tunnel agent must be started by the devcontainer's `postStartCommand`, or a restarted Codespace comes up
|
| 1172 |
+
with `agent_connected: false`.
|
| 1173 |
+
|
| 1174 |
+
### Historical context
|
| 1175 |
+
|
| 1176 |
+
The superseded design ran inference on an **HF Space with ZeroGPU** behind a **Railway** gateway. The
|
| 1177 |
+
active design moves to **Render + Codespace**, CPU-first, with an outbound tunnel. The four-endpoint
|
| 1178 |
+
contract, the gateway responsibility table, the env-var vocabulary and the config freeze are unchanged β
|
| 1179 |
+
only host names moved. `configs/deploy.yaml` still describes the old HF-Space/ZeroGPU target and is left
|
| 1180 |
+
**undisturbed as frozen paperwork** (editing it would move the config hash); no Gradio runtime exists in
|
| 1181 |
+
code. The declared ZeroGPU durations are transcribed, not invented β `app/space_app.py` carries
|
| 1182 |
+
`GPU_DURATIONS` = `vqa` 20, `caption` 20, `grounding` 45, `change` 30, `optical_sar` 45, `change_vqa` 30,
|
| 1183 |
+
and a task with no declared duration is a programming error rather than a default, because silently
|
| 1184 |
+
picking one would reserve the wrong amount of the 5 GPU-minute daily budget. The decoration has **never
|
| 1185 |
+
executed** here (`spaces` is not installed in this environment); `configs/deploy.yaml` sets
|
| 1186 |
+
`cpu_mode_required: true`, so a CPU run must work, and it does.
|
| 1187 |
|
| 1188 |
## Reproducibility
|
| 1189 |
|
| 1190 |
1. **Configuration.** `configs/base.yaml` is the single registry; no magic numbers in Python. Its hash
|
| 1191 |
+
is recorded in every execution trace. Frozen hash: `78f1e3700da15aa1`. A config edit moves the hash and
|
| 1192 |
+
invalidates every artifact keyed to it.
|
| 1193 |
2. **Backbones.** Pinned by `repo_id` + `revision`, never by floating tag:
|
| 1194 |
`HuggingFaceTB/SmolVLM-500M-Instruct@a7da5b986cb5`,
|
| 1195 |
`chendelong/RemoteCLIP@bf1d8a3ccf2d`,
|
|
|
|
| 1202 |
5. **Prompts** are versioned files, frozen before benchmark evaluation.
|
| 1203 |
6. **Negative results are preserved.** Rejected and open rulings are recorded, not removed.
|
| 1204 |
|
| 1205 |
+
The frozen contract, in the registry's own vocabulary:
|
| 1206 |
+
|
| 1207 |
+
| Guarantee | How it is enforced |
|
| 1208 |
+
|---|---|
|
| 1209 |
+
| Frozen configuration | all tunables live in `configs/base.yaml`; the loader validates invariants and computes a hash |
|
| 1210 |
+
| Frozen config hash | `78f1e3700da15aa1`; every artifact records the hash it was produced against |
|
| 1211 |
+
| Pinned backbones | every backbone is pinned by revision; the Hub resolves the exact commit |
|
| 1212 |
+
| Seed | `project.seed: 42` |
|
| 1213 |
+
| Immutable public test | `evaluation.immutable_public_test: true`; `hidden_data_access: false` |
|
| 1214 |
+
| Byte-verified artifacts | every released artifact ships with a sha256 in `models/checksums.sha256` |
|
| 1215 |
+
| Verified metrics | every quoted number is checked against its artifact by the metric-verification tool |
|
| 1216 |
+
|
| 1217 |
+
### Reproduce the metric check (cheap, no GPU)
|
| 1218 |
+
|
| 1219 |
+
```bash
|
| 1220 |
+
python release/tools/verify_readme_metrics.py
|
| 1221 |
+
```
|
| 1222 |
+
|
| 1223 |
+
It **reads** the artifacts under `artifacts/`, **compares** each of the 20 quoted metrics at the precision
|
| 1224 |
+
printed in this README, and **also asserts** the statuses (that the VLM headline contains
|
| 1225 |
+
`ACCEPTANCE-REJECTED`; the router's `corpus_limited` / `n_val`; the calibration temperature and
|
| 1226 |
+
`ece_improvement`). It **exits 0** and prints `ALL CLAIMS VERIFIED` only when everything matches.
|
| 1227 |
+
|
| 1228 |
+
### What "reproduce" means, and what it does not
|
| 1229 |
+
|
| 1230 |
+
| Artifact | Where it trains | Reproducible from this release? |
|
| 1231 |
+
|---|---|---|
|
| 1232 |
+
| router adapter | local CPU | yes β `configs/base.yaml` Β§`router.training` |
|
| 1233 |
+
| grounding head | local | yes β `configs/base.yaml` Β§`grounding_training` |
|
| 1234 |
+
| change head | local | yes β `configs/base.yaml` Β§`change` |
|
| 1235 |
+
| optical_sar fusion head | local, seed sweep | yes (see [`docs/TRAINING.md`](docs/TRAINING.md) Β§5) |
|
| 1236 |
+
| change_vqa head | **external GPU (Kaggle)** | **partly** β the promotion gate, evaluation and serving wiring are reproducible; there is no one-command retrain |
|
| 1237 |
+
| vlm LoRA adapter | **external GPU** | **partly** β same |
|
| 1238 |
+
|
| 1239 |
+
For the two externally-trained artifacts, the repository reproduces the **promotion gate**
|
| 1240 |
+
(byte-identity, sha256, zero non-finite tensors), the **evaluation**, and the **serving wiring**; it does
|
| 1241 |
+
**not** ship a one-command retrain. That is stated rather than implied.
|
| 1242 |
+
|
| 1243 |
+
### What is NOT reproducible from this release
|
| 1244 |
+
|
| 1245 |
+
| Item | Reason |
|
| 1246 |
+
|---|---|
|
| 1247 |
+
| The private deployment repos | they are private; the deployed sources are not in this release |
|
| 1248 |
+
| System-level end-to-end benchmark | **no such benchmark exists** |
|
| 1249 |
+
| Router test-split number | **not run** |
|
| 1250 |
+
| CDVQA / SECOND imagery | public but large; the release documents the acquisition + name-verification procedure, not the data |
|
| 1251 |
+
| BigEarthNet full corpus | **not downloaded** (only a 28k S2 subset was used) |
|
| 1252 |
+
| The historical ZeroGPU/Gradio deploy target | frozen paperwork only; no runtime exists in code |
|
| 1253 |
+
|
| 1254 |
+
Environment traps worth recording for anyone reproducing: the authoring sandbox has a **dead proxy**
|
| 1255 |
+
(outbound calls need `--noproxy '*'` for curl or `ProxyHandler({})` for Python); pytest is installed only
|
| 1256 |
+
in the repository virtualenv (`.venv/Scripts/python.exe`); the full-suite run trips the sandbox's
|
| 1257 |
+
bulk-delete guard; Cloudflare 308-redirects `X.html` β `/X`; and Chrome drops synthetic CDP key events
|
| 1258 |
+
when the window lacks OS focus, which is relevant to any browser-driven reproduction of the live
|
| 1259 |
+
validation.
|
| 1260 |
+
|
| 1261 |
## Known limitations
|
| 1262 |
|
| 1263 |
1. **No system-level end-to-end benchmark exists.** Per-specialist metrics are real; a single
|
|
|
|
| 1267 |
modality accounting in the response confirms the right channels reached the fusion head, but the
|
| 1268 |
presentation is not user-facing.
|
| 1269 |
4. **VQA is weak-but-related.** Asked what terrain dominates a scene, it answers "Grassland".
|
| 1270 |
+
5. **Fusion macro_F1 is low (0.434161)** against 0.931 accuracy β rare classes are poorly handled. Of
|
| 1271 |
+
the 19 classes, 5 are absent from the scored split and contribute 0.0 to macro-F1 by construction.
|
| 1272 |
+
6. **Grounding IoU is modest** (0.2838 canonical, 0.2566 matched6) β useful, not solved β and it is
|
| 1273 |
+
protocol-sensitive: the argmax decode (0.1215) is barely above the zero-shot baseline (0.0972).
|
| 1274 |
+
7. **Calibration makes ECE slightly worse** (0.013755 β 0.014929), and is retained only because it is
|
| 1275 |
+
part of the frozen configuration. The calibrated reliability curve is not plotted.
|
| 1276 |
8. **Router lexical residuals.** *"What is the new runway?"* reads `change` rather than `vqa` (the
|
| 1277 |
`new`-as-change heuristic fires outside `where` questions), and *"How much built-up area was
|
| 1278 |
added?"* reads `vqa` (under-trigger). A lexical router cannot cleanly separate "the new X" from
|
| 1279 |
"what's new"; a trained intent router exists in `artifacts/router/` but is not attached.
|
| 1280 |
+
9. **B-07 tunnel gaps are not fixed in production** (see [Deployment caveats](#deployment-caveats)).
|
| 1281 |
10. **No license has been selected** for this repository. Until one is, the artifacts carry
|
| 1282 |
`license: unknown` and no reuse rights should be assumed. This is an open owner decision.
|
| 1283 |
11. **The Anatomy of a Run page** renders a recorded run whose plate uses the 720Γ720 variant of an
|
| 1284 |
image analysed at 730Γ730 β identical content, scaled by the canvas, but the "actual analysed
|
| 1285 |
image" wording is slightly loose.
|
| 1286 |
+
12. **The BigEarthNet local subset is single-label** (100 %) against the official 1β11 multi-label
|
| 1287 |
+
scheme, so its metrics are **not comparable** to published numbers.
|
| 1288 |
+
13. **The VLM adapter is not accepted** β metrics usable (exact_match 0.963), status
|
| 1289 |
+
acceptance-rejected; the deployed path uses the unadapted model.
|
| 1290 |
+
14. **The deployment repos are private**, so their links 404 for an outside audience β by design.
|
| 1291 |
+
|
| 1292 |
+
### Explicit non-claims
|
| 1293 |
+
|
| 1294 |
+
- **No claim of state-of-the-art performance** on any benchmark.
|
| 1295 |
+
- **No claim of production readiness** for model quality β the deployment runs, but the models carry the
|
| 1296 |
+
limitations above.
|
| 1297 |
+
- **No claim that the trained heads generalise** beyond their training-family test splits.
|
| 1298 |
+
- **No claim that calibration improves confidence.**
|
| 1299 |
+
- **No claim that the VLM adapter is accepted** for production use.
|
| 1300 |
+
- **No system-level accuracy** is claimed anywhere, and none is produced by averaging the per-task
|
| 1301 |
+
numbers.
|
| 1302 |
|
| 1303 |
## Links
|
| 1304 |
|
|
|
|
| 1318 |
| [`antofuller/CROMA`](https://huggingface.co/antofuller/CROMA) | `0dd28e3d633b` | optical/SAR fusion encoder |
|
| 1319 |
|
| 1320 |
Datasets referenced by the evaluations: LEVIR-CD-256 (change), VRSBench (grounding),
|
| 1321 |
+
BigEarthNet (optical-SAR fusion, 19 CLC classes), CDVQA + SECOND (change-VQA). No dataset is
|
| 1322 |
+
redistributed here.
|
| 1323 |
+
|
| 1324 |
+
Documentation in this repository:
|
| 1325 |
+
|
| 1326 |
+
| Document | Contents |
|
| 1327 |
+
|---|---|
|
| 1328 |
+
| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | architecture hub + index of the ten sub-documents |
|
| 1329 |
+
| [`docs/architecture/01-system-overview.md`](docs/architecture/01-system-overview.md) | thesis, component inventory, frozen backbones |
|
| 1330 |
+
| [`docs/architecture/02-deployment-topology.md`](docs/architecture/02-deployment-topology.md) | four tiers, tunnel, wake flow, cold start |
|
| 1331 |
+
| [`docs/architecture/03-request-lifecycle.md`](docs/architecture/03-request-lifecycle.md) | nine-state controller, validation, modality inference, tiling |
|
| 1332 |
+
| [`docs/architecture/04-router.md`](docs/architecture/04-router.md) | MiniLM, five heads, `interpret()` vs `chooseTask()`, the lexical fallback |
|
| 1333 |
+
| [`docs/architecture/05-specialists.md`](docs/architecture/05-specialists.md) | all six tasks end to end |
|
| 1334 |
+
| [`docs/architecture/06-evidence-and-confidence.md`](docs/architecture/06-evidence-and-confidence.md) | evidence schema, aggregation, temperature scaling, the eight events |
|
| 1335 |
+
| [`docs/architecture/07-configuration-freeze.md`](docs/architecture/07-configuration-freeze.md) | the registry, invariants, the config hash |
|
| 1336 |
+
| [`docs/architecture/08-api-contract.md`](docs/architecture/08-api-contract.md) | four endpoints, envelopes, error codes |
|
| 1337 |
+
| [`docs/architecture/09-frontend.md`](docs/architecture/09-frontend.md) | static pages, the Analyze console, real-vs-preview |
|
| 1338 |
+
| [`docs/architecture/10-observability-and-ops.md`](docs/architecture/10-observability-and-ops.md) | health, counters, traces |
|
| 1339 |
+
| [`docs/MODELS.md`](docs/MODELS.md) | the six artifacts in detail; rejected decisions |
|
| 1340 |
+
| [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | headline metrics and the rules they follow |
|
| 1341 |
+
| [`docs/EVALUATION.md`](docs/EVALUATION.md) | per-task protocols; evaluation-honesty rules |
|
| 1342 |
+
| [`docs/DATASETS.md`](docs/DATASETS.md) | measured corpus figures and caveats |
|
| 1343 |
+
| [`docs/TRAINING.md`](docs/TRAINING.md) | per-artifact hyperparameters |
|
| 1344 |
+
| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | live revisions, env vars, traps |
|
| 1345 |
+
| [`docs/REPRODUCIBILITY.md`](docs/REPRODUCIBILITY.md) | what a third party can reproduce |
|
| 1346 |
+
| [`docs/RESEARCH_NOTES.md`](docs/RESEARCH_NOTES.md) | findings, the router defect, negative results |
|
| 1347 |
+
| [`docs/LIMITATIONS.md`](docs/LIMITATIONS.md) | the full limitation catalogue |
|
| 1348 |
+
| [`docs/CHANGELOG.md`](docs/CHANGELOG.md) | versioned record |
|
| 1349 |
+
| [`MODEL_CARD.md`](MODEL_CARD.md) | the Hugging Face model card |
|
| 1350 |
|
| 1351 |
## Citation
|
| 1352 |
|
|
|
|
| 1365 |
## License
|
| 1366 |
|
| 1367 |
**Not yet selected.** See limitation 10. Backbone models remain under their own upstream licenses.
|
| 1368 |
+
|
| 1369 |
+
No `LICENSE` file exists in this repository. Until one is selected, the released artifacts carry
|
| 1370 |
+
`license: unknown` and **no reuse rights should be assumed**. This is an open owner decision, recorded
|
| 1371 |
+
as `OPEN` in [`docs/LIMITATIONS.md`](docs/LIMITATIONS.md) Β§5 and
|
| 1372 |
+
[`docs/CHANGELOG.md`](docs/CHANGELOG.md). The six trained artifacts are small modules over frozen
|
| 1373 |
+
backbones; the backbones are not redistributed here and remain under their own upstream licences β
|
| 1374 |
+
consult each backbone's Hugging Face page.
|
| 1375 |
+
|