gate_reports/gate_1_postmortem.md

Gate 1 Post-Mortem — Subspace validity FAIL

Date: 2026-06-18 Gate: 1 (subspace validity) — the load-bearing, "most likely failure point" per IMPLEMENTATION_SPEC. Outcome: FAIL. Neither label-free lesion-subspace construction localizes LIDC nodules better than a CLS-attention saliency comparator; both barely beat random.

What was tested

Token-level lesion-membership AUROC on held-out LIDC val patches (916,496 patches over 4,676 slices: 2,338 lesion-bearing + 2,338 sampled negatives; lesion prevalence 0.48%). Patch masks were materialized fresh from TCIA DICOM-SEG (z-ordered, image+mask written together — see jobs/materialize_lidc_masks_job.py), because the original eryon manifest's per-slice nodule positions were found to be misaligned with the authoritative SEG under every slice ordering.

Scores compared, all from the frozen MedDINOv3 ViT-B/16 features:

• Construction A: density / kNN-sparse (mean k-NN distance to the 2.1M-token CT bank).
• Construction B: normal-manifold residual (‖(I − UUᵀ)z‖, U = top-64 PCA of the bank).
• Comparator: CLS-to-patch attention from the last block (exact, captured from SDPA).
• Comparator: random.

Result

Scorer	AUROC	95% CI (DeLong)
Attention-saliency	0.767	[0.761, 0.773]
Construction A (density)	0.565	[0.558, 0.572]
Construction B (residual)	0.551	[0.544, 0.558]
Random	0.511	[0.503, 0.520]

DeLong paired differences: A − attention = −0.202 [−0.209, −0.194]; B − attention = −0.216 [−0.223, −0.208]. Both exclude 0 in the wrong direction (p ≈ 0).

Dice@q0.9 vs mask (weak, but > random proxy ~0.005): A 0.079 (>3-patch) / 0.023 (1–3 patch); B 0.067 / 0.020.

Interpretation

The central, explicitly-flagged assumption — the label-free lesion subspace L(x) localizes lesions without labels in MedDINOv3 feature space — does not hold as constructed. Both the density-sparsity prior (Construction A) and the normal-manifold-residual prior (Construction B) carry only weak lesion signal (AUROC ~0.55–0.56), and are decisively beaten by the simplest supervised-free saliency the spec named as the comparator.

The attention comparator scoring 0.767 on the same patches confirms the evaluation is sound (masks, patch rasterization, and alignment are correct) — so this is a true property of the constructions, not an eval artifact. The likely mechanism: in CT, "rare / low-density / high-residual" tokens are dominated by non-lesion rarities (body-boundary, air–tissue interfaces, vessels, motion) rather than nodules, so geometric rarity is not specific to pathology. Attention, by contrast, is shaped by the SSL objective toward salient structure.

Per IMPLEMENTATION_SPEC Gate 1 ("If FAIL: Stop. The method reduces to generic coverage regularization."): the constrained token-economy contribution rests on L(x) being lesion- specific. With L(x) non-specific, the coverage floor would protect generic rare-token directions, not lesions — so the headline Gate 3 claim (beating saliency on small-lesion miss rate) is unlikely, and saliency is in fact the stronger localizer here.

Status of the negative result

This is a clean, publishable negative result of the kind the spec anticipates ("when does coverage-constrained pruning fail, and why"): on a frozen medical SSL backbone, label-free geometric lesion subspaces (density-sparse / normal-residual) do not localize lesions competitively with attention saliency, undermining the premise that motivates a coverage floor.

Options for the human (Gate 1 GO/NO-GO)

1. Accept the FAIL / write up the negative result. Spec-compliant default: stop the direction here (cheapest place to die) and publish the negative finding.
2. Authorize bounded construction variants before final NO-GO (an explicit deviation, not threshold-tuning): e.g. lesion-subspace projection-energy score ‖P_L z‖ instead of raw density/residual; background/air-token masking before density estimation; rank/α/τ sweep; or a hybrid (residual × attention). These probe whether the failure is the prior or its operationalization. None may touch labels (subspace stays label-free).
3. Pivot the comparison framing to "coverage floor on the attention-defined salient set" — but that abandons the label-free-geometry novelty and is effectively a different paper.

No further phases proceed without an explicit human decision (HALT).