gate_reports/gate_1_postmortem.md

# Gate 1 Post-Mortem — Subspace validity FAIL

**Date:** 2026-06-18
**Gate:** 1 (subspace validity) — the load-bearing, "most likely failure point" per IMPLEMENTATION_SPEC.
**Outcome:** FAIL. Neither label-free lesion-subspace construction localizes LIDC nodules
better than a CLS-attention saliency comparator; both barely beat random.

## What was tested

Token-level lesion-membership AUROC on held-out LIDC val patches (916,496 patches over 4,676
slices: 2,338 lesion-bearing + 2,338 sampled negatives; lesion prevalence 0.48%). Patch masks
were materialized fresh from TCIA DICOM-SEG (z-ordered, image+mask written together — see
`jobs/materialize_lidc_masks_job.py`), because the original eryon manifest's per-slice nodule
positions were found to be misaligned with the authoritative SEG under every slice ordering.

Scores compared, all from the frozen MedDINOv3 ViT-B/16 features:
- Construction A: density / kNN-sparse (mean k-NN distance to the 2.1M-token CT bank).
- Construction B: normal-manifold residual (‖(I − UUᵀ)z‖, U = top-64 PCA of the bank).
- Comparator: CLS-to-patch attention from the last block (exact, captured from SDPA).
- Comparator: random.

## Result

| Scorer | AUROC | 95% CI (DeLong) |
|---|---|---|
| Attention-saliency | **0.767** | [0.761, 0.773] |
| Construction A (density) | 0.565 | [0.558, 0.572] |
| Construction B (residual) | 0.551 | [0.544, 0.558] |
| Random | 0.511 | [0.503, 0.520] |

DeLong paired differences: A − attention = −0.202 [−0.209, −0.194]; B − attention =
−0.216 [−0.223, −0.208]. Both exclude 0 in the **wrong** direction (p ≈ 0).

Dice@q0.9 vs mask (weak, but > random proxy ~0.005): A 0.079 (>3-patch) / 0.023 (1–3 patch);
B 0.067 / 0.020.

## Interpretation

The central, explicitly-flagged assumption — *the label-free lesion subspace L(x) localizes
lesions without labels in MedDINOv3 feature space* — does not hold as constructed. Both the
density-sparsity prior (Construction A) and the normal-manifold-residual prior (Construction B)
carry only weak lesion signal (AUROC ~0.55–0.56), and are decisively beaten by the simplest
supervised-free saliency the spec named as the comparator.

The attention comparator scoring 0.767 on the **same** patches confirms the evaluation is
sound (masks, patch rasterization, and alignment are correct) — so this is a true property of
the constructions, not an eval artifact. The likely mechanism: in CT, "rare / low-density /
high-residual" tokens are dominated by non-lesion rarities (body-boundary, air–tissue
interfaces, vessels, motion) rather than nodules, so geometric rarity is not specific to
pathology. Attention, by contrast, is shaped by the SSL objective toward salient structure.

Per IMPLEMENTATION_SPEC Gate 1 ("If FAIL: Stop. The method reduces to generic coverage
regularization."): the constrained token-economy contribution rests on L(x) being lesion-
specific. With L(x) non-specific, the coverage floor would protect generic rare-token
directions, not lesions — so the headline Gate 3 claim (beating saliency on small-lesion miss
rate) is unlikely, and saliency is in fact the stronger localizer here.

## Status of the negative result

This is a clean, publishable negative result of the kind the spec anticipates ("when does
coverage-constrained pruning fail, and why"): on a frozen medical SSL backbone, label-free
geometric lesion subspaces (density-sparse / normal-residual) do **not** localize lesions
competitively with attention saliency, undermining the premise that motivates a coverage floor.

## Options for the human (Gate 1 GO/NO-GO)

1. **Accept the FAIL / write up the negative result.** Spec-compliant default: stop the
   direction here (cheapest place to die) and publish the negative finding.
2. **Authorize bounded construction variants before final NO-GO** (an explicit deviation, not
   threshold-tuning): e.g. lesion-subspace **projection-energy** score ‖P_L z‖ instead of raw
   density/residual; background/air-token masking before density estimation; rank/α/τ sweep;
   or a hybrid (residual × attention). These probe whether the failure is the *prior* or its
   *operationalization*. None may touch labels (subspace stays label-free).
3. **Pivot the comparison framing** to "coverage floor on the attention-defined salient set"
   — but that abandons the label-free-geometry novelty and is effectively a different paper.

No further phases proceed without an explicit human decision (HALT).