Datasets

Every figure below was measured from the data on disk, not copied from a paper or a dataset schema. Where a corpus was only partially acquired, or where the local subset differs from the official release, that is stated — not smoothed over.

Status tags: ACQUIRED · MEASURED · PARTIAL · NOT DOWNLOADED · REJECTED.

1. Summary

Task	Dataset	Role	Local state
`change`	LEVIR-CD-256	train / val / test	ACQUIRED — split 7120 / 1024 / 2048
`change_vqa`	CDVQA (+ SECOND imagery)	train / val / test	annotations + imagery ACQUIRED; name-verified 2,968/2,968
`grounding`	VRSBench	eval (16,159 records)	ACQUIRED — boxes normalised 0–100
`optical_sar`	BigEarthNet (CLC-19)	train / held-out test	PARTIAL — 28,000-patch local subset
`vlm`	BigEarthNet (instruction pairs)	LoRA adaptation	PARTIAL — same 28k subset
—	SECOND (SCD)	CDVQA imagery source	ACQUIRED, extracted to `data/cdvqa/{im1,im2,label1,label2}/`

2. LEVIR-CD-256 — change detection

The standard change-detection benchmark, 256×256 tiles. Split used (from configs/base.yaml):

Split	Count
train	7,120
val	1,024
test	2,048

Measured test-split detail (artifacts/change/eval_test/eval_result.json):

Property	Value
n	2,048
images containing change	935
mean change fraction	0.0509 (≈ 5 % of pixels)
threshold	0.50
tile size	256
device (eval)	cuda
total pixels scored	134,217,728

The 5 % change fraction is why pooled and macro metrics are both reported: with a strong class imbalance, pooled IoU (0.8122) and macro IoU (0.8457) answer different questions. The per-pixel confusion counts (tp 5,978,997 / fp 523,658 / fn 858,407 / tn 126,856,666) are stored so any metric can be recomputed rather than trusted.

3. CDVQA (+ SECOND) — change-VQA

The CDVQA repository publishes annotations only — no imagery. The imagery is publicly available as SECOND (SCD). This project acquired SECOND, name-verified it (2,968 / 2,968 MATCH), extracted it to data/cdvqa/{im1,im2,label1,label2}/, and verified a CDVQA example loads against it end-to-end.

Measured corpus figures:

Property	Value
Val images	400
Val questions	16,441
Test questions	39,686
Test2 questions	second held-out set (accuracy 0.651469)

Temporal and label semantics — established from evidence, with honest uncertainty:

Mapping	State
`label1` = pre, `label2` = post	established
`im1` = pre, `im2` = post	supported, not proven

Corpus root: data/cdvqa — the directory that contains annotations/. Passing data/cdvqa/annotations is rejected by _resolve_root (verified by execution: it raises no 'annotations' directory under data\cdvqa\annotations). The correct call is load_cdvqa('data/cdvqa').

Not established: trainability of the raw loader. The adapter is pair-aware and require_images=True succeeds, but the raw corpus has no training loop of its own — the shipped change_vqa head trains on cached features, not on the raw loader. See TRAINING.md.

4. VRSBench — grounding evaluation

The grounding head is evaluated on VRSBench. Measured: 16,159 eval records.

VRSBench stores boxes normalised to 0–100; this project stores boxes normalised to 0–1. The conversion is declared explicitly as grounding.benchmark_box_scale: 100.0 so it cannot be applied twice or forgotten.

Because the box convention is a common source of silent error, grounding is reported under two protocols (canonical and matched6) and two decode variants — see BENCHMARKS.md §1.

5. BigEarthNet — optical-SAR fusion and VLM adaptation

BigEarthNet is used in two places: as the label space and evaluation benchmark for the optical-SAR fusion head, and as the instruction-pair source for the SmolVLM LoRA adaptation.

Measured local subset:

Property	Value
S2 patches (local subset)	28,000
tiles	98
bands per patch	12
full official corpus	480,038 patches
label space	19 CLC classes

5.1 Label-semantics caveat (important)

The local 28k subset is 100 % single-label, against the official 1–11 multi-label scheme. Metrics computed on this subset are therefore not comparable to published multi-label BigEarthNet numbers. Any statement of the form "BigEarthNet mAP = X" is false for this subset.

5.2 Fusion evaluation detail

Measured (artifacts/optical_sar/fusion_head_production_v001/pre_registered_115_metric.json):

Property	Value
split	test
n scored	4,000
classes in label space	19
classes present in the scored split	14
classes absent	5
macro-F1 denominator	all 19 classes (absent classes contribute 0.0)
accuracy	0.931
macro F1	0.434161
deciding statistic	False

The macro-F1 denominator is recorded explicitly so a reader can see that absent classes drag the macro score down by construction. This is why accuracy (0.931) and macro-F1 (0.434161) must be read together.

5.3 The BigEarthNet data-format contradiction

PARTIALLY — reported, not silently resolved. The BigEarthNet data format contradicts the original plan. This was reported rather than quietly patched, because silently changing the preprocessing would move the frozen config hash. See RESEARCH_NOTES.md for the full record.

The BigEarthNet documentation — its uses, mentions, or endorsements — does not specify a percentile stretch. This project nevertheless applies percentile normalisation for optical inputs (2/98) to match the CROMA contract. That is a deliberate, documented choice, not an upstream fact.

6. Data hygiene and leakage controls

Splits are by group, never by example, for the router: template / hard-negative families are kept whole, and hard-negative families are placed in the test split so their accuracy measures generalisation rather than memorisation.
Leakage split key: scene_id (declared in configs/base.yaml).
Immutable public test: true. hidden_data_access: false. The evaluation config forbids touching hidden data.

7. What is NOT available

Corpus	State
BigEarthNet S1+S2 full corpus	NOT DOWNLOADED (only the 28k S2 subset is local)
BigEarthNet multi-label (reBEN) results	not produced — the local subset is single-label
Cross-dataset generalisation sets	not used
Any private / hidden evaluation data	not accessed (`hidden_data_access: false`)