Buckets:

Hsyshasuzh
/

longmemory-geo

Files

xet

Hsyshasuzh/longmemory-geo / project /ROADMAP_AND_PROGRESS.md

Hsyshasuzh

about 7 hours ago

preview code

download

raw

16.6 kB

# LongMemory-Geo roadmap and progress

Updated: 2026-11-05

This is the operational handoff for building a high-quality private
multi-task geospatial perception model. It supplements `PROJECT_STATE.md`;
it does not overwrite historical evidence, test reports, or checkpoints.

## Current execution roadmap — 99% target

The researched plan is `ROADMAP_99_PERCENT.md`; primary sources are recorded in
`docs/reviews/2026-11-05-99-roadmap-sources.md` and hashed local evidence is in
`evaluation/runs/2026-11-05-99-roadmap-evidence.json`.

This supersedes the next-action interpretation below: correct and audit sampling
and evaluation before further model experiments. The class-balanced sampler uses
coverage/source-image labels instead of eligible tile positives in the historical
runs, the run named
rehearsal900 actually used 812 DOTA tiles, and all recent runs completed one
epoch. Their candidate checkpoints regressed, but these runs do not establish
that more data or class balancing cannot help after convergence. The measured
best retained SODA-A development checkpoint remains unpromoted.

Phase 0 progress: the repaired sampler now derives class membership from the
same clipped/ignored targets consumed by training and logs replacement exposure;
SODA-A maps both airplane and helicopter to aircraft through their distinct
reviewed detector head classes. The full train/validation-only audit is
`evaluation/runs/2026-11-05-soda-a-supervision-audit.json`: 25,446 train and
13,740 validation tiles, with 12,927/8,822 unique eligible aircraft source
annotations, respectively. It read no official test annotations or imagery.
Adapter plus detection/evaluation regression suites passed (41 broader tests).
The remaining Phase 0 gates are ignore-policy sensitivity, mining’s one-to-one
matching repair, and a full-scene baseline; do not start another training run
until those measurement checks are complete.

The first two remaining checks are now complete: the ignore-policy sensitivity
report is `evaluation/runs/2026-11-05-soda-a-ignore-policy-sensitivity.json`,
and the regenerated six-class DOTA train-only review queue is
`evaluation/runs/2026-11-05-dota-train-review-queue-v2.json` with sheets in
the matching `*-sheets-v2/` directory. It is pending source-label review and
is not training data. The next gate is the full-scene baseline.

The user's target is broad satellite coverage, including correctly labeled
objects. Existing six-class detection cannot certify every object category.
Pursue broad coverage and require an explicit, supported all-profile/class
acceptance gate; never describe a finite benchmark pass as a universal guarantee.

## 2026-11-05 SODA-A validation frontier

- VERIFIED: added `evaluation/score_soda_a_frontier.py`, which evaluates only
  the validation tile IDs recorded by a SODA-A training run and never
  materializes the official test split.
- VERIFIED: evaluated
  `training/runs/2026-11-04-soda-a-small-baseline/best.pt` on all 60 recorded
  SODA-A validation tiles using the reviewed small OBB source checkpoint,
  native 1024 inference, CUDA T4, and the existing 50% ignore-overlap policy.
  The report reproduces the training metrics: mAP50-95 `0.3869758267`, AP50
  `0.7344298656`, precision `0.4921524664`, recall `0.9070247934`.
- VERIFIED: the empirical same-validation 99%-precision frontier is vehicle
  recall `0.2733`, large-vehicle recall `0.1944`, aircraft recall `0.7010`,
  vessel has no feasible threshold, and storage-tank recall `0.6433`. SODA-A
  contains no bridge targets, so this run provides no bridge evidence.
- VERIFIED: initial evaluator output was discarded as invalid because it
  counted unsupported bridge-head predictions. The corrected evaluator filters
  to SODA-A's exact mapped classes; the corrected report is
  `evaluation/runs/2026-11-05-soda-a-small-baseline-val-v2.json` with raw
  records beside it.
- VERIFIED: `tests/model/test_soda_a_adapter.py` passed and the new evaluator
  passes Python compilation. These are development diagnostics only; no model
  was promoted and the SODA-A official test remains untouched.
- VERIFIED: generated `evaluation/runs/2026-11-05-soda-a-error-summary.json`
  from the retained SODA-A validation and DOTA selection records. SODA-A
  tile-level errors are vehicle 133 FP/23 misses, large vehicle 77/5,
  aircraft 96/4, vessel 58/1, and storage tank 89/12. DOTA selection also
  shows bridge 87 FP/3 misses and storage tank 124 FP/4 misses. These are
  development counts, not qualitative label-error findings.
- VERIFIED: class-balanced sampling was rejected under
  `training/runs/2026-11-05-soda-a-class-balanced/`: mAP50-95 `0.37258` vs
  uniform-450 `0.38698`, with lower same-validation 99%-precision recall for
  every SODA-A class. DOTA rehearsal-900 was also rejected at mAP50-95
  `0.35433`; storage-tank frontier recall improved modestly, but vehicle fell
  to `0.1570`. Increasing the SODA-A pool to 1,200 tiles was rejected at
  mAP50-95 `0.37447`, with vehicle frontier recall collapsing to `0.0581` even
  though storage-tank recall rose to `0.6959`. None of these checkpoints is
  promoted.
- INFERRED: the current short-run bottleneck is not simply class balance,
  DOTA rehearsal quantity, or more SODA-A tiles under the fixed 900-second
  budget. Qualitative review of high-confidence false positives and misses is
  required before another training variable is selected.
- VERIFIED: localized representative high-confidence false positives and
  missed targets by class in
  `evaluation/runs/2026-11-05-detection-error-locations.json`. The report is
  based only on retained SODA-A validation and DOTA selection records and does
  not alter labels or use protected test data.
- VERIFIED: reviewed both rendered error sheets. The observations and limits
  are recorded in `docs/reviews/2026-11-05-soda-a-error-review.md`; they
  identify vehicle/vessel, bridge/linear-infrastructure, and tank/industrial
  structure confusion hypotheses without treating them as label corrections.
- VERIFIED: mined 812 DOTA training tiles with the retained uniform SODA-A
  candidate to create a source-label review queue at confidence >= `0.5`:
  20 vehicle, 20 large-vehicle, 20 vessel, 12 storage-tank, and 7 bridge
  candidates. The queue is
  `evaluation/runs/2026-11-05-dota-train-hard-negative-review-queue.json` and
  is explicitly ineligible for training until source-label review.

## Product direction

The target is a practical RGB overhead-imagery model with a shared backbone
and three task heads:

1. **Oriented-object detection (OBB):** bridge, vehicle, large vehicle,
   aircraft, vessel, and storage tank.
2. **Semantic segmentation:** building, road, water, vegetation,
   agricultural area, bare ground, and developed area.
3. **Optional scene context:** scene-level classification or representation
   learning. This is where PatternNet belongs; it must not be converted into
   fabricated object boxes.

The target is to outperform selected public baselines on a fixed, documented
benchmark contract. “Better than every public model” is not a meaningful claim
without specifying datasets, target imagery, precision/recall operating points,
hardware, and latency budget.

## Non-negotiable measurement rules

- Keep train, selection-validation, and final test imagery separate.
- Preserve geographic/source grouping; do not train on quarantined audit data.
- Select checkpoints and thresholds only on validation data.
- Report per-class precision, recall, AP50, mAP50-95, object-size results,
  false-positive/false-negative evidence, latency, and memory.
- A 99% precision/99% recall assertion requires every supported class to meet
  both targets on a frozen, untouched, adequately supported test set. Training
  loss, reused development scenes, and in-sample fitted thresholds do not
  establish the claim.

## Dataset roles

| Dataset | Role | Task | Current status |
| --- | --- | --- | --- |
| Existing DOTA corpus | Rehearsal and historical diagnostics | OBB detection | Existing source; retain rehearsal to reduce forgetting. |
| FAIR1M mirror | Adaptation/development | OBB detection | 1,732 paired images; footprint-overlap groups are enforced; audit groups quarantined. Geographic independence remains unresolved. |
| SODA-A | New detector source | OBB detection | Full archive acquired and adapter/trainer implemented. Official test split remains untouched. |
| Existing OpenEarthMap corpus | Semantic training/evaluation | Segmentation | Shipped segmenter is the best accepted semantic candidate. |
| PatternNet | Optional future context/pretraining task | Scene classification | Not detector supervision: image-level labels, no object boxes. |

### SODA-A contract

- Official source records: 2,513 images.
- Official split inventory: 1,067 train, 576 validation, 870 test images.
- Derived native 1024px tiles: 25,446 train, 13,740 validation, and 20,755
  test tiles.
- Exact detector mappings: airplane→aircraft, small-vehicle→vehicle,
  large-vehicle→large vehicle, ship→vessel, storage-tank→storage tank.
- Unsupported SODA-A categories, official ignore areas, and malformed
  non-quadrilateral annotations remain ignored regions. They are never made
  into background or artificial OBB labels.

## Implemented continuation work (2026-11-04)

### SODA-A acquisition and adapter

- Downloaded `datasets/external/soda-a/raw/Annotations.zip` and
  `Images.zip`.
- Added `training/soda_a.py`:
  - validates official image/annotation pairing;
  - reads the flat image archive and official split annotations;
  - produces padded 1024px source tiles with source-coordinate OBBs;
  - preserves partial/ignored regions;
  - retains SODA-A test data only for future frozen evaluation.
- Extended `training/detection_data.py` so SODA-A ignored categories use the
  same explicit source-ignore contract as FAIR1M/DIOR.
- Added `tests/model/test_soda_a_adapter.py`.
- Focused SODA-A plus FAIR1M adapter tests passed.
- Real GPU smoke passed on an annotated SODA-A tile: finite loss `2.7731`,
  12 supervised positives, and peak allocation approximately `1,386.5 MiB`
  on the T4.

### SODA-A bounded baseline

Run: `training/runs/2026-11-04-soda-a-small-baseline/`

- Detector: reviewed `yolo26s-obb` small architecture.
- Training: 600 SODA-A tiles + 450 DOTA rehearsal tiles.
- Validation: 60 official SODA-A validation tiles.
- Resolution: 1024; batch 1; AdamW; LR `3e-6`; frozen BatchNorm statistics.
- Budget: 900 seconds; one full epoch completed; status `TIME_LIMIT`.
- SODA-A test split was not loaded for model selection, threshold fitting, or
  training.

| Metric | Epoch 0 | Epoch 1 |
| --- | ---: | ---: |
| SODA-A validation mAP50-95 | 0.2998 | 0.3870 |
| SODA-A validation AP50 | 0.6108 | 0.7344 |
| Low-threshold precision | 0.4701 | 0.4922 |
| Low-threshold recall | 0.7459 | 0.9070 |
| Mean training loss | — | 1.2995 |

The epoch-1 checkpoint is the current SODA-A development candidate. It is not
promoted.

### Cross-dataset diagnostic for the SODA-A candidate

Checkpoint: `training/runs/2026-11-04-soda-a-small-baseline/best.pt`

Evidence:
`evaluation/runs/2026-11-04-soda-a-small-baseline-dota.json` and retained raw
records in `evaluation/runs/2026-11-04-soda-a-small-baseline-dota-records.json`.

| Diagnostic | mAP50-95 | AP50 | Low-threshold precision | Low-threshold recall |
| --- | ---: | ---: | ---: | ---: |
| DOTA selection (development only) | 0.5702 | 0.8141 | 0.5473 | 0.9647 |
| Fixed DOTA quick (development only) | 0.5346 | — | 0.5427 | 0.9725 |

Same-selection 99%-precision frontier (optimistic because thresholds are fitted
and scored on the same selection rows):

| Class | Recall at empirical >=99% precision |
| --- | ---: |
| Bridge | No feasible threshold (7 targets) |
| Vehicle | 43.6% |
| Large vehicle | 60.2% |
| Aircraft | 100.0% |
| Vessel | 94.3% |
| Storage tank | 45.1% |

Conclusion: SODA-A adaptation substantially improves low-threshold recall and
does not show catastrophic DOTA collapse, but it does not meet the intended
99% precision/recall objective. Bridges, vehicles, and tanks need targeted
work. These DOTA diagnostics are not independent final benchmarks.

### FAIR1M latest controlled run

Run: `training/runs/2026-11-04-fair1m-small-lr3e-6/`

- Grouped FAIR1M split; historical audit groups quarantined.
- 600 FAIR1M train tiles, 450 DOTA rehearsal tiles, 40 DIOR bridge images,
  200 reviewed DIOR bridge expansions, and 60 grouped FAIR1M validation tiles.
- Small detector; LR `3e-6`; 900-second cap; one epoch completed.

| Metric | Initialization | Epoch 1 |
| --- | ---: | ---: |
| Grouped FAIR1M mAP50-95 | 0.2528 | 0.3141 |
| AP50 | 0.4707 | 0.6204 |
| Low-threshold precision | 0.4387 | 0.3106 |
| Low-threshold recall | 0.5252 | 0.7905 |

This is development evidence only. No FAIR1M checkpoint is promoted.

## Roadmap

### Phase 1 — establish trustworthy evaluation

1. Preserve existing final/test evidence without retuning on it.
2. Build independent, geographically/source-separated evaluation scenes for
   intended deployment sensors and resolutions.
3. Audit duplicates/near-duplicates and source overlap across FAIR1M, SODA-A,
   DOTA, DIOR, and future datasets.
4. Set a frozen benchmark contract: per-class P/R, AP, size strata,
   segmentation IoU, scene accuracy, tiled-image behavior, latency, and RAM.

### Phase 2 — detector data and diagnostics

1. Run a SODA-A validation precision frontier without reading its official
   test split.
2. Generate false-negative/false-positive reports by class, object size,
   density, tile boundary, source, and scene type.
3. Inspect high-confidence errors before changing labels or policies.
4. Use DOTA rehearsal and protected source splits in every mixed-source run.

### Phase 3 — controlled detector experiments

1. Treat the current SODA-A epoch-1 checkpoint as the baseline.
2. Test exactly one variable at a time, beginning with class-balanced/hard
   negative sampling for vehicle, bridge, and tank errors.
3. Compare native 1024px tiles with a justified high-resolution or multiscale
   alternative only on fixed validation scenes.
4. Compare small vs nano or another architecture only with matching data,
   seed, budget, and evaluation rows.
5. Freeze thresholds on selection data and score once on untouched test data.

### Phase 4 — segmentation and scene context

1. Retain the shipped semantic segmenter as the current accepted baseline.
2. Add new segmentation data only with validated class mapping and geographic
   separation.
3. Evaluate PatternNet separately as scene-context supervision or backbone
   pretraining; do not merge image-level labels into detector loss.
4. Add a scene head only if it improves measured downstream context or
   uncertainty handling without harming detector/segmenter quality.

### Phase 5 — large-image inference and release

1. Benchmark tile size, overlap, fusion, and seam behavior on full scenes.
2. Keep oriented NMS/fusion and source-coordinate transformations under unit
   and qualitative tests.
3. Package frozen weights, configs, thresholds, dataset provenance, metrics,
   and failure analysis.
4. Publish a model card that distinguishes validated claims from development
   diagnostics.

## Immediate next actions

1. Render and review the train-only queue against original source annotations;
   approve only unambiguous negatives and preserve uncertain examples.
2. Design one hard-negative/confusion-focused run from approved train-only
   records; retain the uniform 600 SODA-A / 450 DOTA baseline, architecture,
   LR, validation IDs, and budget.
3. Run and evaluate that one-variable experiment; prioritize
   reducing high-confidence vehicle, large-vehicle, tank, and bridge false
   positives while preserving recall. Do not train on
   SODA-A test or FAIR1M audit groups.
3. Keep PatternNet isolated until a scene-head or pretraining experiment is
   explicitly designed and benchmarked.

## Known limitations

- The project has no certified independent geographic evaluation suite yet.
- Current 99%-precision frontiers are selection-fitted diagnostics, not final
  reliability estimates.
- FAIR1M footprint and cross-source dependence remain unresolved.
- SODA-A’s official test is correctly protected but has not yet been used for
  a single frozen final evaluation.
- PatternNet provides scene labels only; it cannot supply detector boxes.
- The detector ONNX export still lacks strict parity; use PyTorch detector
  inference for validated results.

Xet Storage Details

Size:: 16.6 kB
Xet hash:: fa2453d6040d5821dc946fbe4eefece152b294df22a91225cb2074a1217045ef

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# LongMemory-Geo roadmap and progress

	Updated: 2026-11-05

	This is the operational handoff for building a high-quality private
	multi-task geospatial perception model. It supplements `PROJECT_STATE.md`;
	it does not overwrite historical evidence, test reports, or checkpoints.

	## Current execution roadmap — 99% target

	The researched plan is `ROADMAP_99_PERCENT.md`; primary sources are recorded in
	`docs/reviews/2026-11-05-99-roadmap-sources.md` and hashed local evidence is in
	`evaluation/runs/2026-11-05-99-roadmap-evidence.json`.

	This supersedes the next-action interpretation below: correct and audit sampling
	and evaluation before further model experiments. The class-balanced sampler uses
	coverage/source-image labels instead of eligible tile positives in the historical
	runs, the run named
	rehearsal900 actually used 812 DOTA tiles, and all recent runs completed one
	epoch. Their candidate checkpoints regressed, but these runs do not establish
	that more data or class balancing cannot help after convergence. The measured
	best retained SODA-A development checkpoint remains unpromoted.

	Phase 0 progress: the repaired sampler now derives class membership from the
	same clipped/ignored targets consumed by training and logs replacement exposure;
	SODA-A maps both airplane and helicopter to aircraft through their distinct
	reviewed detector head classes. The full train/validation-only audit is
	`evaluation/runs/2026-11-05-soda-a-supervision-audit.json`: 25,446 train and
	13,740 validation tiles, with 12,927/8,822 unique eligible aircraft source
	annotations, respectively. It read no official test annotations or imagery.
	Adapter plus detection/evaluation regression suites passed (41 broader tests).
	The remaining Phase 0 gates are ignore-policy sensitivity, mining’s one-to-one
	matching repair, and a full-scene baseline; do not start another training run
	until those measurement checks are complete.

	The first two remaining checks are now complete: the ignore-policy sensitivity
	report is `evaluation/runs/2026-11-05-soda-a-ignore-policy-sensitivity.json`,
	and the regenerated six-class DOTA train-only review queue is
	`evaluation/runs/2026-11-05-dota-train-review-queue-v2.json` with sheets in
	the matching `*-sheets-v2/` directory. It is pending source-label review and
	is not training data. The next gate is the full-scene baseline.

	The user's target is broad satellite coverage, including correctly labeled
	objects. Existing six-class detection cannot certify every object category.
	Pursue broad coverage and require an explicit, supported all-profile/class
	acceptance gate; never describe a finite benchmark pass as a universal guarantee.

	## 2026-11-05 SODA-A validation frontier

	- VERIFIED: added `evaluation/score_soda_a_frontier.py`, which evaluates only
	the validation tile IDs recorded by a SODA-A training run and never
	materializes the official test split.
	- VERIFIED: evaluated
	`training/runs/2026-11-04-soda-a-small-baseline/best.pt` on all 60 recorded
	SODA-A validation tiles using the reviewed small OBB source checkpoint,
	native 1024 inference, CUDA T4, and the existing 50% ignore-overlap policy.
	The report reproduces the training metrics: mAP50-95 `0.3869758267`, AP50
	`0.7344298656`, precision `0.4921524664`, recall `0.9070247934`.
	- VERIFIED: the empirical same-validation 99%-precision frontier is vehicle
	recall `0.2733`, large-vehicle recall `0.1944`, aircraft recall `0.7010`,
	vessel has no feasible threshold, and storage-tank recall `0.6433`. SODA-A
	contains no bridge targets, so this run provides no bridge evidence.
	- VERIFIED: initial evaluator output was discarded as invalid because it
	counted unsupported bridge-head predictions. The corrected evaluator filters
	to SODA-A's exact mapped classes; the corrected report is
	`evaluation/runs/2026-11-05-soda-a-small-baseline-val-v2.json` with raw
	records beside it.
	- VERIFIED: `tests/model/test_soda_a_adapter.py` passed and the new evaluator
	passes Python compilation. These are development diagnostics only; no model
	was promoted and the SODA-A official test remains untouched.
	- VERIFIED: generated `evaluation/runs/2026-11-05-soda-a-error-summary.json`
	from the retained SODA-A validation and DOTA selection records. SODA-A
	tile-level errors are vehicle 133 FP/23 misses, large vehicle 77/5,
	aircraft 96/4, vessel 58/1, and storage tank 89/12. DOTA selection also
	shows bridge 87 FP/3 misses and storage tank 124 FP/4 misses. These are
	development counts, not qualitative label-error findings.
	- VERIFIED: class-balanced sampling was rejected under
	`training/runs/2026-11-05-soda-a-class-balanced/`: mAP50-95 `0.37258` vs
	uniform-450 `0.38698`, with lower same-validation 99%-precision recall for
	every SODA-A class. DOTA rehearsal-900 was also rejected at mAP50-95
	`0.35433`; storage-tank frontier recall improved modestly, but vehicle fell
	to `0.1570`. Increasing the SODA-A pool to 1,200 tiles was rejected at
	mAP50-95 `0.37447`, with vehicle frontier recall collapsing to `0.0581` even
	though storage-tank recall rose to `0.6959`. None of these checkpoints is
	promoted.
	- INFERRED: the current short-run bottleneck is not simply class balance,
	DOTA rehearsal quantity, or more SODA-A tiles under the fixed 900-second
	budget. Qualitative review of high-confidence false positives and misses is
	required before another training variable is selected.
	- VERIFIED: localized representative high-confidence false positives and
	missed targets by class in
	`evaluation/runs/2026-11-05-detection-error-locations.json`. The report is
	based only on retained SODA-A validation and DOTA selection records and does
	not alter labels or use protected test data.
	- VERIFIED: reviewed both rendered error sheets. The observations and limits
	are recorded in `docs/reviews/2026-11-05-soda-a-error-review.md`; they
	identify vehicle/vessel, bridge/linear-infrastructure, and tank/industrial
	structure confusion hypotheses without treating them as label corrections.
	- VERIFIED: mined 812 DOTA training tiles with the retained uniform SODA-A
	candidate to create a source-label review queue at confidence >= `0.5`:
	20 vehicle, 20 large-vehicle, 20 vessel, 12 storage-tank, and 7 bridge
	candidates. The queue is
	`evaluation/runs/2026-11-05-dota-train-hard-negative-review-queue.json` and
	is explicitly ineligible for training until source-label review.

	## Product direction

	The target is a practical RGB overhead-imagery model with a shared backbone
	and three task heads:

	1. Oriented-object detection (OBB): bridge, vehicle, large vehicle,
	aircraft, vessel, and storage tank.
	2. Semantic segmentation: building, road, water, vegetation,
	agricultural area, bare ground, and developed area.
	3. Optional scene context: scene-level classification or representation
	learning. This is where PatternNet belongs; it must not be converted into
	fabricated object boxes.

	The target is to outperform selected public baselines on a fixed, documented
	benchmark contract. “Better than every public model” is not a meaningful claim
	without specifying datasets, target imagery, precision/recall operating points,
	hardware, and latency budget.

	## Non-negotiable measurement rules

	- Keep train, selection-validation, and final test imagery separate.
	- Preserve geographic/source grouping; do not train on quarantined audit data.
	- Select checkpoints and thresholds only on validation data.
	- Report per-class precision, recall, AP50, mAP50-95, object-size results,
	false-positive/false-negative evidence, latency, and memory.
	- A 99% precision/99% recall assertion requires every supported class to meet
	both targets on a frozen, untouched, adequately supported test set. Training
	loss, reused development scenes, and in-sample fitted thresholds do not
	establish the claim.

	## Dataset roles

	\| Dataset \| Role \| Task \| Current status \|
	\| --- \| --- \| --- \| --- \|
	\| Existing DOTA corpus \| Rehearsal and historical diagnostics \| OBB detection \| Existing source; retain rehearsal to reduce forgetting. \|
	\| FAIR1M mirror \| Adaptation/development \| OBB detection \| 1,732 paired images; footprint-overlap groups are enforced; audit groups quarantined. Geographic independence remains unresolved. \|
	\| SODA-A \| New detector source \| OBB detection \| Full archive acquired and adapter/trainer implemented. Official test split remains untouched. \|
	\| Existing OpenEarthMap corpus \| Semantic training/evaluation \| Segmentation \| Shipped segmenter is the best accepted semantic candidate. \|
	\| PatternNet \| Optional future context/pretraining task \| Scene classification \| Not detector supervision: image-level labels, no object boxes. \|

	### SODA-A contract

	- Official source records: 2,513 images.
	- Official split inventory: 1,067 train, 576 validation, 870 test images.
	- Derived native 1024px tiles: 25,446 train, 13,740 validation, and 20,755
	test tiles.
	- Exact detector mappings: airplane→aircraft, small-vehicle→vehicle,
	large-vehicle→large vehicle, ship→vessel, storage-tank→storage tank.
	- Unsupported SODA-A categories, official ignore areas, and malformed
	non-quadrilateral annotations remain ignored regions. They are never made
	into background or artificial OBB labels.

	## Implemented continuation work (2026-11-04)

	### SODA-A acquisition and adapter

	- Downloaded `datasets/external/soda-a/raw/Annotations.zip` and
	`Images.zip`.
	- Added `training/soda_a.py`:
	- validates official image/annotation pairing;
	- reads the flat image archive and official split annotations;
	- produces padded 1024px source tiles with source-coordinate OBBs;
	- preserves partial/ignored regions;
	- retains SODA-A test data only for future frozen evaluation.
	- Extended `training/detection_data.py` so SODA-A ignored categories use the
	same explicit source-ignore contract as FAIR1M/DIOR.
	- Added `tests/model/test_soda_a_adapter.py`.
	- Focused SODA-A plus FAIR1M adapter tests passed.
	- Real GPU smoke passed on an annotated SODA-A tile: finite loss `2.7731`,
	12 supervised positives, and peak allocation approximately `1,386.5 MiB`
	on the T4.

	### SODA-A bounded baseline

	Run: `training/runs/2026-11-04-soda-a-small-baseline/`

	- Detector: reviewed `yolo26s-obb` small architecture.
	- Training: 600 SODA-A tiles + 450 DOTA rehearsal tiles.
	- Validation: 60 official SODA-A validation tiles.
	- Resolution: 1024; batch 1; AdamW; LR `3e-6`; frozen BatchNorm statistics.
	- Budget: 900 seconds; one full epoch completed; status `TIME_LIMIT`.
	- SODA-A test split was not loaded for model selection, threshold fitting, or
	training.

	\| Metric \| Epoch 0 \| Epoch 1 \|
	\| --- \| ---: \| ---: \|
	\| SODA-A validation mAP50-95 \| 0.2998 \| 0.3870 \|
	\| SODA-A validation AP50 \| 0.6108 \| 0.7344 \|
	\| Low-threshold precision \| 0.4701 \| 0.4922 \|
	\| Low-threshold recall \| 0.7459 \| 0.9070 \|
	\| Mean training loss \| — \| 1.2995 \|

	The epoch-1 checkpoint is the current SODA-A development candidate. It is not
	promoted.

	### Cross-dataset diagnostic for the SODA-A candidate

	Checkpoint: `training/runs/2026-11-04-soda-a-small-baseline/best.pt`

	Evidence:
	`evaluation/runs/2026-11-04-soda-a-small-baseline-dota.json` and retained raw
	records in `evaluation/runs/2026-11-04-soda-a-small-baseline-dota-records.json`.

	\| Diagnostic \| mAP50-95 \| AP50 \| Low-threshold precision \| Low-threshold recall \|
	\| --- \| ---: \| ---: \| ---: \| ---: \|
	\| DOTA selection (development only) \| 0.5702 \| 0.8141 \| 0.5473 \| 0.9647 \|
	\| Fixed DOTA quick (development only) \| 0.5346 \| — \| 0.5427 \| 0.9725 \|

	Same-selection 99%-precision frontier (optimistic because thresholds are fitted
	and scored on the same selection rows):

	\| Class \| Recall at empirical >=99% precision \|
	\| --- \| ---: \|
	\| Bridge \| No feasible threshold (7 targets) \|
	\| Vehicle \| 43.6% \|
	\| Large vehicle \| 60.2% \|
	\| Aircraft \| 100.0% \|
	\| Vessel \| 94.3% \|
	\| Storage tank \| 45.1% \|

	Conclusion: SODA-A adaptation substantially improves low-threshold recall and
	does not show catastrophic DOTA collapse, but it does not meet the intended
	99% precision/recall objective. Bridges, vehicles, and tanks need targeted
	work. These DOTA diagnostics are not independent final benchmarks.

	### FAIR1M latest controlled run

	Run: `training/runs/2026-11-04-fair1m-small-lr3e-6/`

	- Grouped FAIR1M split; historical audit groups quarantined.
	- 600 FAIR1M train tiles, 450 DOTA rehearsal tiles, 40 DIOR bridge images,
	200 reviewed DIOR bridge expansions, and 60 grouped FAIR1M validation tiles.
	- Small detector; LR `3e-6`; 900-second cap; one epoch completed.

	\| Metric \| Initialization \| Epoch 1 \|
	\| --- \| ---: \| ---: \|
	\| Grouped FAIR1M mAP50-95 \| 0.2528 \| 0.3141 \|
	\| AP50 \| 0.4707 \| 0.6204 \|
	\| Low-threshold precision \| 0.4387 \| 0.3106 \|
	\| Low-threshold recall \| 0.5252 \| 0.7905 \|

	This is development evidence only. No FAIR1M checkpoint is promoted.

	## Roadmap

	### Phase 1 — establish trustworthy evaluation

	1. Preserve existing final/test evidence without retuning on it.
	2. Build independent, geographically/source-separated evaluation scenes for
	intended deployment sensors and resolutions.
	3. Audit duplicates/near-duplicates and source overlap across FAIR1M, SODA-A,
	DOTA, DIOR, and future datasets.
	4. Set a frozen benchmark contract: per-class P/R, AP, size strata,
	segmentation IoU, scene accuracy, tiled-image behavior, latency, and RAM.

	### Phase 2 — detector data and diagnostics

	1. Run a SODA-A validation precision frontier without reading its official
	test split.
	2. Generate false-negative/false-positive reports by class, object size,
	density, tile boundary, source, and scene type.
	3. Inspect high-confidence errors before changing labels or policies.
	4. Use DOTA rehearsal and protected source splits in every mixed-source run.

	### Phase 3 — controlled detector experiments

	1. Treat the current SODA-A epoch-1 checkpoint as the baseline.
	2. Test exactly one variable at a time, beginning with class-balanced/hard
	negative sampling for vehicle, bridge, and tank errors.
	3. Compare native 1024px tiles with a justified high-resolution or multiscale
	alternative only on fixed validation scenes.
	4. Compare small vs nano or another architecture only with matching data,
	seed, budget, and evaluation rows.
	5. Freeze thresholds on selection data and score once on untouched test data.

	### Phase 4 — segmentation and scene context

	1. Retain the shipped semantic segmenter as the current accepted baseline.
	2. Add new segmentation data only with validated class mapping and geographic
	separation.
	3. Evaluate PatternNet separately as scene-context supervision or backbone
	pretraining; do not merge image-level labels into detector loss.
	4. Add a scene head only if it improves measured downstream context or
	uncertainty handling without harming detector/segmenter quality.

	### Phase 5 — large-image inference and release

	1. Benchmark tile size, overlap, fusion, and seam behavior on full scenes.
	2. Keep oriented NMS/fusion and source-coordinate transformations under unit
	and qualitative tests.
	3. Package frozen weights, configs, thresholds, dataset provenance, metrics,
	and failure analysis.
	4. Publish a model card that distinguishes validated claims from development
	diagnostics.

	## Immediate next actions

	1. Render and review the train-only queue against original source annotations;
	approve only unambiguous negatives and preserve uncertain examples.
	2. Design one hard-negative/confusion-focused run from approved train-only
	records; retain the uniform 600 SODA-A / 450 DOTA baseline, architecture,
	LR, validation IDs, and budget.
	3. Run and evaluate that one-variable experiment; prioritize
	reducing high-confidence vehicle, large-vehicle, tank, and bridge false
	positives while preserving recall. Do not train on
	SODA-A test or FAIR1M audit groups.
	3. Keep PatternNet isolated until a scene-head or pretraining experiment is
	explicitly designed and benchmarked.

	## Known limitations

	- The project has no certified independent geographic evaluation suite yet.
	- Current 99%-precision frontiers are selection-fitted diagnostics, not final
	reliability estimates.
	- FAIR1M footprint and cross-source dependence remain unresolved.
	- SODA-A’s official test is correctly protected but has not yet been used for
	a single frozen final evaluation.
	- PatternNet provides scene labels only; it cannot supply detector boxes.
	- The detector ONNX export still lacks strict parity; use PyTorch detector
	inference for validated results.