thundercode commited on
Commit
b10b360
Β·
verified Β·
1 Parent(s): bf13032

release: add README.md

Browse files
Files changed (1) hide show
  1. README.md +491 -1
README.md CHANGED
@@ -1,3 +1,493 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
- license: unknown
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SatQuery AI
2
+
3
+ **An interactive vision-language assistant for multimodal remote-sensing image analysis.**
4
+
5
+ Ask a natural-language question about a satellite or aerial image β€” or a pair of images β€” and SatQuery
6
+ routes it to the right specialist models, collects evidence, and returns a single confidence-scored
7
+ result envelope. It runs on CPU, is served from a static frontend, and is live at
8
+ **https://satquery.pages.dev**.
9
+
10
+ <p align="center">
11
+ <img src="screenshots/analyze-grounding.png" alt="The SatQuery Analyze console answering a grounding question against live inference" width="820">
12
+ </p>
13
+
14
+ > **Status: research prototype, pre-1.0.** The architecture is frozen. This repository is the
15
+ > documented public release of the system, its trained artifacts, and its measured results β€”
16
+ > **including the negative ones.**
17
+
18
  ---
19
+
20
+ ## Table of contents
21
+
22
+ - [Motivation](#motivation)
23
+ - [What the system supports](#what-the-system-supports)
24
+ - [Supported inputs](#supported-inputs)
25
+ - [Architecture](#architecture)
26
+ - [Routing and the execution trace](#routing-and-the-execution-trace)
27
+ - [Real inference vs. the preview path](#real-inference-vs-the-preview-path)
28
+ - [Models](#models)
29
+ - [Measured results](#measured-results)
30
+ - [Live validation](#live-validation)
31
+ - [Installation](#installation)
32
+ - [Local development](#local-development)
33
+ - [Deployment](#deployment)
34
+ - [Reproducibility](#reproducibility)
35
+ - [Known limitations](#known-limitations)
36
+ - [Links](#links)
37
+ - [Citation](#citation)
38
+
39
  ---
40
+
41
+ ## Motivation
42
+
43
+ Remote-sensing analysis is fragmented. Detecting change between two acquisitions, localising an
44
+ object, captioning a scene, answering a question about it, and fusing optical with SAR each live in a
45
+ different model, a different preprocessing convention, and a different output schema. Assembling them
46
+ into one answer means re-solving the same problems β€” tiling, band handling, coordinate systems,
47
+ confidence β€” every time.
48
+
49
+ SatQuery AI explores a single hypothesis: **a small deterministic router plus a shared evidence
50
+ contract can make a heterogeneous specialist ensemble behave like one system**, without a large
51
+ language model in the control path. The router *understands* the query. A deterministic policy
52
+ *decides* which specialists run. The specialists *compute*. The evidence engine *proves* the answer.
53
+
54
+ Two design rules follow from that, and they are non-negotiable in the codebase:
55
+
56
+ - **No LLM-generated coordinates. No LLM-generated confidence.** Coordinates come from detection and
57
+ segmentation heads; confidence comes from a calibrated scoring path.
58
+ - **Every result carries an observable execution trace** β€” never chain-of-thought.
59
+
60
+ ## What the system supports
61
+
62
+ Six specialist tasks. All six are reported `available: true` by the live capability contract
63
+ (`GET /api/capabilities`, probed 2026-09-25).
64
+
65
+ | Task | What it answers | Assets |
66
+ |---|---|---|
67
+ | `vqa` | A free-form question about a single scene | 1 |
68
+ | `caption` | A description of a single scene | 1 |
69
+ | `grounding` | *Where* is a described object or region β€” returns boxes | 1 |
70
+ | `change` | *What changed* between two co-registered acquisitions β€” returns change regions | 2 |
71
+ | `change_vqa` | A yes/no or short question about a detected change | 2 |
72
+ | `optical_sar` | Joint scene classification from an optical + SAR pair | 2 |
73
+
74
+ ## Supported inputs
75
+
76
+ Confirmed by the implementation, not assumed:
77
+
78
+ | Modality | Task(s) | Format |
79
+ |---|---|---|
80
+ | Optical, single image | `vqa`, `caption`, `grounding` | JPEG, PNG, TIFF |
81
+ | Temporal optical pair | `change`, `change_vqa` | Two images of **identical dimensions** |
82
+ | Optical + SAR pair | `optical_sar` | GeoTIFF/TIFF preferred |
83
+
84
+ **Modality is inferred server-side from band count**, not from the file extension: `{1, 2}` bands β‡’
85
+ SAR, `{3, 4, 8, 11, 12, 13}` bands β‡’ optical. The browser cannot read band count, so the console warns
86
+ when a submitted pair looks like two ordinary photographs rather than an optical/SAR pair.
87
+
88
+ **Per-file upload limit: 4,194,304 bytes (4 MiB).** Larger files are refused with HTTP 413 β€” imagery
89
+ must be downscaled first.
90
+
91
+ ## Architecture
92
+
93
+ This is the **actually deployed** topology. An older direct-client-to-inference design is superseded.
94
+
95
+ ```mermaid
96
+ flowchart TD
97
+ B["Browser<br/>(static console)"] -->|HTTPS| CF["Cloudflare Pages<br/>satquery.pages.dev"]
98
+ CF -->|"HTTPS JSON Β· /api/*"| R["Render<br/>satquery-orchestrator"]
99
+ R -->|"outbound long-poll<br/>POST /tunnel/agent"| T{{"outbound tunnel"}}
100
+ T --> C["GitHub Codespace<br/>FastAPI inference Β· CPU Β· :8000"]
101
+ C --> S["Specialists"]
102
+ S --> M["SmolVLM Β· RemoteCLIP Β· STANet-change<br/>CROMA-fusion Β· MiniLM router"]
103
+ M --> E["Evidence engine<br/>+ temperature scaling"]
104
+ E --> RE["ResultEnvelope"]
105
+ RE -->|"tunnel β†’ Render"| B
106
+ ```
107
+
108
+ **Why a tunnel.** The inference host runs in a GitHub Codespace. The forwarded-port path is not
109
+ reachable for a private repo (it returns HTTP 302), so the orchestrator keeps a **long-poll tunnel**:
110
+ the Codespace dials out to `POST /tunnel/agent` and holds the connection; Render queues work onto it.
111
+ `transport_mode` is `auto`, and the tunnel is the live transport.
112
+
113
+ ### Repository map
114
+
115
+ | Path | Contents |
116
+ |---|---|
117
+ | `app/` | FastAPI inference service and its composition root |
118
+ | `core/` | Config registry, evidence engine, contracts |
119
+ | `specialists/` | One module per specialist (vqa, caption, grounding, change, optical_sar) |
120
+ | `router/` | MiniLM intent router |
121
+ | `gateway/` | Render orchestration hub (`/api/*`, CORS, wake flow) |
122
+ | `frontend/` | The static console (HTML/CSS/JS) |
123
+ | `configs/base.yaml` | The frozen configuration registry β€” single source of truth |
124
+ | `evaluation/`, `training/` | Evaluation harnesses and training entry points |
125
+ | `artifacts/` | Trained heads, checkpoints, evaluation outputs, provenance |
126
+ | `docs/` | Architecture, models, benchmarks, deployment, limitations |
127
+
128
+ ## Routing and the execution trace
129
+
130
+ Routing is deliberately two-stage, and the split matters:
131
+
132
+ 1. **`interpret()` β€” the reading.** A lexical pass over the query produces a *reading*: task intent,
133
+ modality, temporal requirement, spatial scope, and expected evidence kind. It is
134
+ **asset-count-blind**.
135
+ 2. **`chooseTask()` β€” the dispatch.** The reading is combined with the number of attached assets to
136
+ decide the task actually dispatched. This is why a reading of `change` with **one** asset dispatches
137
+ `change_vqa` β€” the documented quantifier upgrade.
138
+
139
+ > **A router bug worth recording.** An earlier revision evaluated the temporal rule before the
140
+ > location rule, so *"Where are the built-up areas in this image?"* matched `\bbuilt\b` as a *change*
141
+ > marker and `area` inside *"areas"* as a quantifier. With one asset it collapsed to `vqa` and
142
+ > answered "River". Fixed on 2026-09-25 in `frontend/assets/js/mission.js`; the fix is covered by
143
+ > regression tests and verified live. The same defect existed on a second surface
144
+ > (`SQ.policy` in `core.js`) and was fixed the same day.
145
+
146
+ ### The eight execution events
147
+
148
+ The console renders an execution trace built from **eight events**, emitted by the frontend around
149
+ real network calls (`SQ.EVENT_NAMES` in `frontend/assets/js/core.js`):
150
+
151
+ | # | Event | Emitted when |
152
+ |---|---|---|
153
+ | 1 | `QUERY_RECEIVED` | The query and assets are accepted |
154
+ | 2 | `QUERY_UNDERSTOOD` | `interpret()` has produced the reading |
155
+ | 3 | `ROUTE_SELECTED` | `chooseTask()` has selected the dispatched task |
156
+ | 4 | `SPECIALIST_STARTED` | The inference request has been issued |
157
+ | 5 | `SPECIALIST_COMPLETED` | The specialist has returned |
158
+ | 6 | `EVIDENCE_GENERATED` | Evidence items are available |
159
+ | 7 | `CONFIDENCE_COMPUTED` | The calibrated confidence is available |
160
+ | 8 | `RESULT_ASSEMBLED` | The `ResultEnvelope` is complete |
161
+
162
+ These are a **frontend** vocabulary driven by observable events β€” not a backend protocol and not a
163
+ model's reasoning trace. On live runs the trace bar reaches **94.4444 %** (17/18) and every node is
164
+ marked live; the preview path is the only source of mock-marked nodes.
165
+
166
+ ## Real inference vs. the preview path
167
+
168
+ - **Real path (production).** With assets attached, the console calls
169
+ `POST /api/infer` on the Render orchestrator. Every run returns a real `run_*` identifier from the
170
+ inference service. **Live validation recorded 0 mock nodes across 24 live runs.**
171
+ - **Preview path.** With *no* files selected, the console renders a labelled illustrative preview so
172
+ the interface is explorable without the stack awake. Preview nodes are explicitly marked `is-mock`
173
+ and never appear in a live run.
174
+
175
+ The distinction is observable, not asserted: a live run shows `live Β· N evidence Β· transport …` and
176
+ zero `.trace__node.is-mock` elements.
177
+
178
+ ## Models
179
+
180
+ Six trained artifacts are released. **Four are task heads and two are adapters** β€” none is a complete
181
+ standalone model, and each documents its backbone dependency. Full detail: [`docs/MODELS.md`](docs/MODELS.md),
182
+ [`MODEL_CARD.md`](MODEL_CARD.md), and the generated [`models/manifest.json`](models/manifest.json).
183
+
184
+ | Task | Backbone (pinned) | Custom component | Artifact | Size | Eval data | Metric | Status |
185
+ |---|---|---|---|---|---|---|---|
186
+ | `change` | STANet-style, ResNet-18 encoder, PAM | change head | `head.pt` | 63,231,009 B | LEVIR-CD-256, test n=2048 | pooled IoU **0.8122** Β· macro IoU **0.8457** Β· pooled F1 **0.8964** | **VERIFIED** |
187
+ | `grounding` | `chendelong/RemoteCLIP` ViT-B/32 @ `bf1d8a3ccf2d` (frozen) | trainable head (2048β†’512) | `head.pt` | 12,639,041 B | VRSBench, n=16159 | mean_best_IoU **0.2838** Β· recall@0.5 **0.2198** (canonical) | measured β€” two protocols |
188
+ | `optical_sar` | `antofuller/CROMA` base @ `0dd28e3d633b` | fusion head (2318β†’512β†’19) | `head.pt` | 14,427,457 B | BigEarthNet, 19 CLC classes, test n=4000 | accuracy **0.931** Β· macro_F1 **0.434161** | measured β€” **ruling OPEN** |
189
+ | `change_vqa` | as `change` | change-VQA head | `head.pt` | 5,822,809 B | test n=39686 | accuracy **0.697626** Β· macro_F1 **0.378373** | measured β€” **ruling OPEN** |
190
+ | `router` | `sentence-transformers/all-MiniLM-L6-v2` @ `1110a243fdf4` | intent adapter | `adapter.pt` | 211,961 B | val n=86 | accuracy **0.965116** | **TEST NOT RUN** |
191
+ | `vqa` / `caption` | `HuggingFaceTB/SmolVLM-500M-Instruct` @ `a7da5b986cb5` | **LoRA** (r=16, Ξ±=32, dropout 0.05) | `adapter_model.safetensors` | 34,798,048 B | frozen 1000-Q subset | exact_match **0.963** Β· F1 **0.96432** | **ACCEPTANCE-REJECTED** |
192
+
193
+ Backbones are third-party and pinned by `repo_id` + `revision` in `configs/base.yaml`; they are
194
+ fetched from the Hugging Face Hub, not redistributed here.
195
+
196
+ ## Measured results
197
+
198
+ Every number below traces to an artifact, a test, or a live run. **Nothing here is a system-level
199
+ benchmark β€” no such benchmark exists** (see [Known limitations](#known-limitations)).
200
+
201
+ | Metric | Value | Split / protocol | Source key | Status |
202
+ |---|---|---|---|---|
203
+ | Change pooled IoU | 0.8122 | LEVIR-CD-256 test, n=2048, thr 0.50 | `metrics.pooled.iou` | **VERIFIED** |
204
+ | Change macro IoU | 0.8457 | same | `metrics.macro.miou` | **VERIFIED** |
205
+ | Change pooled F1 | 0.8964 | same | `metrics.pooled.f1` | **VERIFIED** |
206
+ | Grounding mean_best_IoU (canonical, `head_threshold`) | 0.2838 | VRSBench, n=16159 | `results.head_threshold.mean_best_iou` | measured |
207
+ | Grounding recall@0.5 (canonical, `head_threshold`) | 0.2198 | same | `results.head_threshold.recall.0.50` | measured |
208
+ | Grounding mean_best_IoU (matched6, `head_threshold`) | 0.2566 | VRSBench, n=16159 | `results.head_threshold.mean_best_iou` | measured |
209
+ | Grounding recall@0.5 (matched6, `head_threshold`) | 0.1938 | same | `results.head_threshold.recall.0.50` | measured |
210
+ | Grounding `head_argmax` decode (both protocols) | 0.1215 | same | `results.head_argmax.mean_best_iou` | measured β€” **worse** |
211
+ | Grounding zero-shot baseline | 0.0972 | same | `results.zero_shot_matched.mean_best_iou` | measured |
212
+ | Optical-SAR accuracy | 0.931 | BigEarthNet, held-out test n=4000 | `accuracy` | measured β€” ruling **OPEN** |
213
+ | Optical-SAR macro_F1 | 0.434161 | same | `macro_f1` | measured β€” ruling **OPEN** |
214
+ | Change-VQA accuracy (**test**) | 0.697626 | test n=39686 | `verification.test_accuracy` | measured β€” ruling **OPEN** |
215
+ | Change-VQA macro_F1 (**test**) | 0.378373 | same | `verification.test_macro_f1` | measured β€” ruling **OPEN** |
216
+ | Change-VQA accuracy (**test2**) | 0.651469 | second test set | `verification.test2_accuracy` | measured β€” **lower** |
217
+ | Change-VQA macro_F1 (**test2**) | 0.372309 | second test set | `verification.test2_macro_f1` | measured β€” **lower** |
218
+ | VLM adapter exact_match | 0.963 | frozen 1000-Q subset | `artifacts/vlm/phase6_closure.json` | USABLE_VERIFIED β€” **ACCEPTANCE-REJECTED** |
219
+ | VLM adapter F1 | 0.96432 | same | same | USABLE_VERIFIED β€” **ACCEPTANCE-REJECTED** |
220
+ | Router **overall ungated** accuracy | 0.965116 | val, n=86, corpus-limited | `overall_ungated_accuracy` | **TEST NOT RUN** |
221
+ | System-level end-to-end benchmark | β€” | β€” | β€” | **NOT RUN β€” none exists** |
222
+
223
+ ### Grounding: three decode variants, two protocols
224
+
225
+ The grounding head is evaluated under **two matching protocols** (canonical, matched6) and **three
226
+ decode variants**. Quoting a single number would misrepresent the result, so all of them are listed:
227
+
228
+ | Decode | canonical mean_best_IoU | matched6 mean_best_IoU |
229
+ |---|---|---|
230
+ | `head_threshold` (the headline number) | **0.2838** | **0.2566** |
231
+ | `head_argmax` | 0.1215 | 0.1215 |
232
+ | `zero_shot_matched` (baseline, no head) | 0.0972 | 0.0972 |
233
+
234
+ The head clears the zero-shot baseline, but only the threshold decode is meaningfully above it β€” the
235
+ argmax decode (0.1215) is barely better than zero-shot. The absolute level is modest either way:
236
+ **grounding is useful, not solved.**
237
+
238
+ ### Fusion: measured but the ruling is open
239
+
240
+ Optical-SAR fusion reaches **0.931 accuracy** on a 19-class held-out set of 4,000 β€” but only
241
+ **0.434 macro_F1**. Those two numbers describe very different things: the model is accurate on
242
+ frequent classes and weak on rare ones. The acceptance ruling for this head is **OPEN**, and the
243
+ headline accuracy must never be quoted without the macro_F1 beside it.
244
+
245
+ ### Change-VQA: two test sets, and they disagree
246
+
247
+ `artifacts/change_vqa/run/PROMOTION.json` records **two** test evaluations:
248
+
249
+ | Split | accuracy | macro_F1 |
250
+ |---|---|---|
251
+ | `test` | 0.697626 | 0.378373 |
252
+ | `test2` | **0.651469** | 0.372309 |
253
+
254
+ The `test` numbers are the higher pair. Both are reported here; quoting only `test` would overstate
255
+ the result. The acceptance ruling is **OPEN**.
256
+
257
+ ### Phase 6 / VLM: deployment success β‰  model acceptance
258
+
259
+ The Phase-6 SmolVLM LoRA adapter reaches **exact_match 0.963** and **F1 0.96432** on a frozen
260
+ 1,000-question subset. It is marked **USABLE_VERIFIED** and **ACCEPTANCE-REJECTED**.
261
+
262
+ Those two verdicts are not in conflict, and the distinction is the point:
263
+
264
+ - **USABLE_VERIFIED** β€” the adapter loads, runs, and produces the measured numbers in the deployed
265
+ pipeline.
266
+ - **ACCEPTANCE-REJECTED** β€” the change did not clear the project's own pre-registered acceptance bar.
267
+
268
+ A model can be a working engineering artifact and a rejected research result at the same time.
269
+ This release keeps both labels.
270
+
271
+ ### Calibration: it got worse, and we say so
272
+
273
+ The `change_vqa` confidence path applies temperature scaling (`T = 0.9773`). Measured on the
274
+ validation split (n=16441):
275
+
276
+ | | ECE |
277
+ |---|---|
278
+ | Before temperature scaling | **0.013755** |
279
+ | After temperature scaling | **0.014929** |
280
+
281
+ **Calibration did not improve β€” it moved slightly worse.** The scaling is retained because it is part
282
+ of the frozen configuration, not because it helped. The reliability curve plotted on the Benchmark page
283
+ is explicitly labelled as the **pre-scaling** diagram so a reader cannot mistake it for the calibrated
284
+ result.
285
+
286
+ ## Live validation
287
+
288
+ Validation drove the **production site** in a headed browser, one upload per case, with per-case
289
+ screenshots and recorded run identifiers.
290
+
291
+ | Property | Result |
292
+ |---|---|
293
+ | Independent full passes | **3** |
294
+ | Cases per pass | 8 (6 regression + 2 defect) |
295
+ | Passes at 8/8 | **3 of 3** |
296
+ | Live runs executed | **24** |
297
+ | Correct dispatches | **24** |
298
+ | Mock-node contamination | **0** on every live run |
299
+ | Trace fill | 94.4444 % on every live run |
300
+ | Frontend regression suite | **106 passed** (`tests/unit/test_frontend_live_wiring.py`) |
301
+
302
+ Each pass produced **fresh run identifiers** β€” no run id is shared between passes.
303
+
304
+ The harness asserts the form state **before** dispatch β€” that the query box really holds the intended
305
+ query, that `#obsTail` reads `ready`, and that both frames are attached for pair tasks. This matters:
306
+ an earlier harness revision typed with synthetic key events that Chrome silently drops when the window
307
+ lacks OS focus, so it dispatched the page's *default* query and still recorded a "result". The
308
+ assertions exist because of that failure.
309
+
310
+ ### Representative real run IDs
311
+
312
+ One full pass (pass 3 of 3) β€” the same pass the screenshots below are drawn from. Run identifiers
313
+ are fresh on every pass; the other two passes recorded different ids.
314
+
315
+ | Case | Query | Dispatched | Run ID |
316
+ |---|---|---|---|
317
+ | vqa | What type of terrain dominates this scene? | `vqa` | `run_fef26e91e7e6` |
318
+ | caption | Describe the main visual characteristics of this scene. | `caption` | `run_96281bdfcc08` |
319
+ | grounding | Where are the visible buildings in this image? | `grounding` | `run_e49adc8d319f` |
320
+ | change | What changed between the earlier and later image? | `change` | `run_aedc59cbcdc9` |
321
+ | change_vqa | Did the coastline advance between the two observations? | `change_vqa` | `run_62ca98d510be` |
322
+ | optical_sar | …combining the optical and SAR observations? | `optical_sar` | `run_beacf6aa4e21` |
323
+ | **grounding** | **Where are the built-up areas in this image?** | **`grounding`** | **`run_467ffa406f22`** |
324
+ | **grounding** | **Where is the new airport?** | **`grounding`** | **`run_46980ba55c62`** |
325
+
326
+ The last two are the router-defect queries. Both previously collapsed to `vqa` and answered "River".
327
+
328
+ ### Screenshots
329
+
330
+ Eight captures from the post-fix live run (headed browser, 1384Γ—855, one upload per case). Each
331
+ panel shows the run identifier, the frozen config hash `78f1e3700da15aa1`, and the evidence list
332
+ returned by the specialist β€” nothing is mocked.
333
+
334
+ | | |
335
+ |---|---|
336
+ | ![Grounding β€” built-up areas](screenshots/analyze-grounding.png) | ![Optical-SAR](screenshots/analyze-optical-sar.png) |
337
+ | **Grounding** β€” "Where are the built-up areas in this image?" β€” the fixed router defect (`run_467ffa406f22`, dispatched `grounding`, not `vqa`) | **Optical-SAR** fusion on a real optical/SAR GeoTIFF pair (`run_beacf6aa4e21`, fused class 18) |
338
+ | ![Grounding β€” new airport](screenshots/analyze-grounding-new-airport.png) | ![Grounding β€” visible buildings](screenshots/analyze-grounding-buildings.png) |
339
+ | **Grounding** β€” "Where is the new airport?" β€” second defect query (`run_46980ba55c62`, dispatched `grounding`) | **Grounding** β€” "Where are the visible buildings in this image?" (`run_e49adc8d319f`) |
340
+ | ![Change](screenshots/analyze-change.png) | ![Caption](screenshots/analyze-caption.png) |
341
+ | **Change** detection on a same-shape temporal pair | **Caption** of a single scene (`run_96281bdfcc08`, calibrated confidence 1.000) |
342
+ | ![VQA](screenshots/analyze-vqa.png) | ![Change-VQA](screenshots/analyze-change-vqa.png) |
343
+ | **VQA** β€” "What type of terrain dominates this scene?" | **Change-VQA** β€” "Did the coastline advance between the two observations?" |
344
+
345
+ All eight are reproduced byte-for-byte in the evidence archive (Phase 7) with SHA-256 recorded in
346
+ `RELEASE_MANIFEST.md`.
347
+
348
+ ## Installation
349
+
350
+ Python 3.11+ and a CPU are sufficient. No CUDA requirement β€” device is selected via
351
+ `SATQUERY_DEVICE`; all placement is `.to(device)`, never `.cuda()`.
352
+
353
+ ```bash
354
+ git clone https://github.com/Anish-lab-blip/SatQuery-AI
355
+ cd SatQuery-AI
356
+ python -m venv .venv
357
+ source .venv/Scripts/activate # Windows git-bash; use .venv/bin/activate on Linux/macOS
358
+ pip install -r requirements.txt
359
+ ```
360
+
361
+ Backbones are fetched from the Hugging Face Hub on first use, pinned by revision in
362
+ `configs/base.yaml`. The frozen config hash is **`78f1e3700da15aa1`** β€” the loader refuses to run a
363
+ config that violates the recorded invariants (for example `fusion.input_dim == 3*encoder_dim + 12 + 2`).
364
+
365
+ ## Local development
366
+
367
+ ```bash
368
+ # Inference service, CPU (this is the launcher the Codespace runs)
369
+ PORT=8000 python deploy/codespace/serve.py
370
+
371
+ # Health
372
+ curl localhost:8000/v1/health
373
+ ```
374
+
375
+ The frontend is fully static and needs no build step to serve locally:
376
+
377
+ ```bash
378
+ python -m http.server 5500 --directory frontend
379
+ ```
380
+
381
+ Run the frontend regression suite:
382
+
383
+ ```bash
384
+ python -m pytest tests/unit/test_frontend_live_wiring.py -q
385
+ ```
386
+
387
+ ## Deployment
388
+
389
+ The live topology is **Cloudflare Pages β†’ Render β†’ outbound tunnel β†’ GitHub Codespace**.
390
+
391
+ | Layer | Role | Source |
392
+ |---|---|---|
393
+ | Cloudflare Pages | Static frontend at **https://satquery.pages.dev** | `frontend/` |
394
+ | Render | Orchestrator / API gateway, `/api/*`, CORS, wake flow | `gateway/` |
395
+ | GitHub Codespace | FastAPI inference host, CPU, port 8000 | `app/` |
396
+ | Hugging Face | Model cards, released artifacts, checksums | this release |
397
+
398
+ Deployment sources are **separate repositories** from this release. The wake flow is: Cloudflare β†’
399
+ Render β†’ start the Codespace if stopped β†’ poll `/v1/health` β†’ surface *"Waking inference engine…"* β†’
400
+ `POST /infer` β†’ result.
401
+
402
+ ### Deployment caveats
403
+
404
+ - **Cold start.** The inference host may be stopped when idle. The first request after a cold start
405
+ can exceed the client timeout while weights are fetched; a retry a few seconds later normally
406
+ succeeds. Warm the stack before any demonstration and confirm
407
+ `GET /api/health` reports `tunnel.agent_connected: true`.
408
+ - **Tunnel gaps.** The tunnel agent can be briefly absent. A request issued during such a gap may hang
409
+ or return HTTP 504. **This is not fixed in production** β€” a prepared patch
410
+ (`forward_unavailable` 503 / `upstream_timeout` 504 plus a `codespace_name` fix) exists and is
411
+ documented, but it was deliberately not deployed. Root cause: in `auto` transport mode a tunnel
412
+ timeout falls through to the forwarded-port path, which then spends the 120 s wake timeout on an
413
+ HTTP 302 β€” the observed ~249 s failure.
414
+ - **`codespace_name`** is still reported with a trailing newline by `/api/health` (cosmetic; the wake
415
+ path strips it).
416
+
417
+ ## Reproducibility
418
+
419
+ 1. **Configuration.** `configs/base.yaml` is the single registry; no magic numbers in Python. Its hash
420
+ is recorded in every execution trace. Frozen hash: `78f1e3700da15aa1`.
421
+ 2. **Backbones.** Pinned by `repo_id` + `revision`, never by floating tag:
422
+ `HuggingFaceTB/SmolVLM-500M-Instruct@a7da5b986cb5`,
423
+ `chendelong/RemoteCLIP@bf1d8a3ccf2d`,
424
+ `sentence-transformers/all-MiniLM-L6-v2@1110a243fdf4`,
425
+ `antofuller/CROMA@0dd28e3d633b`.
426
+ 3. **Released artifacts.** `models/manifest.json` and `models/checksums.sha256` are **generated from
427
+ the actual files** β€” never hand-typed. Verify with `sha256sum -c models/checksums.sha256`.
428
+ 4. **Splits.** LEVIR-CD-256: train 7120 / val 1024 / test 2048. Grounding: VRSBench n=16159.
429
+ Fusion: held-out test n=4000. Change-VQA: test n=39686. Leakage isolation is by `scene_id`.
430
+ 5. **Prompts** are versioned files, frozen before benchmark evaluation.
431
+ 6. **Negative results are preserved.** Rejected and open rulings are recorded, not removed.
432
+
433
+ ## Known limitations
434
+
435
+ 1. **No system-level end-to-end benchmark exists.** Per-specialist metrics are real; a single
436
+ end-to-end number is **NOT RUN**.
437
+ 2. **Router accuracy is validation-only** (n=86, corpus-limited). Its **test set was never run**.
438
+ 3. **Optical-SAR fusion returns a bare class index** (`class_18`), not a human-readable label. The
439
+ modality accounting in the response confirms the right channels reached the fusion head, but the
440
+ presentation is not user-facing.
441
+ 4. **VQA is weak-but-related.** Asked what terrain dominates a scene, it answers "Grassland".
442
+ 5. **Fusion macro_F1 is low (0.434)** against 0.931 accuracy β€” rare classes are poorly handled.
443
+ 6. **Grounding IoU is modest** (0.2838 canonical) β€” useful, not solved.
444
+ 7. **Calibration makes ECE slightly worse**, and is retained only because it is part of the frozen
445
+ configuration.
446
+ 8. **Router lexical residuals.** *"What is the new runway?"* reads `change` rather than `vqa` (the
447
+ `new`-as-change heuristic fires outside `where` questions), and *"How much built-up area was
448
+ added?"* reads `vqa` (under-trigger). A lexical router cannot cleanly separate "the new X" from
449
+ "what's new"; a trained intent router exists in `artifacts/router/` but is not attached.
450
+ 9. **B-07 tunnel gaps are not fixed in production** (see Deployment caveats).
451
+ 10. **No license has been selected** for this repository. Until one is, the artifacts carry
452
+ `license: unknown` and no reuse rights should be assumed. This is an open owner decision.
453
+ 11. **The Anatomy of a Run page** renders a recorded run whose plate uses the 720Γ—720 variant of an
454
+ image analysed at 730Γ—730 β€” identical content, scaled by the canvas, but the "actual analysed
455
+ image" wording is slightly loose.
456
+
457
+ ## Links
458
+
459
+ | | |
460
+ |---|---|
461
+ | Live demo | https://satquery.pages.dev |
462
+ | GitHub | https://github.com/Anish-lab-blip/SatQuery-AI |
463
+ | Hugging Face | https://huggingface.co/thundercode/SatQuery |
464
+
465
+ Third-party models this work builds on (pinned, not redistributed):
466
+
467
+ | Model | Revision | Role |
468
+ |---|---|---|
469
+ | [`HuggingFaceTB/SmolVLM-500M-Instruct`](https://huggingface.co/HuggingFaceTB/SmolVLM-500M-Instruct) | `a7da5b986cb5` | VQA + captioning backbone |
470
+ | [`chendelong/RemoteCLIP`](https://huggingface.co/chendelong/RemoteCLIP) | `bf1d8a3ccf2d` | remote-sensing grounding encoder |
471
+ | [`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | `1110a243fdf4` | router embedding |
472
+ | [`antofuller/CROMA`](https://huggingface.co/antofuller/CROMA) | `0dd28e3d633b` | optical/SAR fusion encoder |
473
+
474
+ Datasets referenced by the evaluations: LEVIR-CD-256 (change), VRSBench (grounding),
475
+ BigEarthNet (optical-SAR fusion, 19 CLC classes). No dataset is redistributed here.
476
+
477
+ ## Citation
478
+
479
+ No paper accompanies this release. Until one exists, cite the repository:
480
+
481
+ ```bibtex
482
+ @misc{satqueryai2026,
483
+ title = {SatQuery AI: an interactive vision-language assistant for
484
+ multimodal remote-sensing image analysis},
485
+ author = {SatQuery AI contributors},
486
+ year = {2026},
487
+ url = {https://github.com/Anish-lab-blip/SatQuery-AI}
488
+ }
489
+ ```
490
+
491
+ ## License
492
+
493
+ **Not yet selected.** See limitation 10. Backbone models remain under their own upstream licenses.