| --- |
| license: mit |
| tags: |
| - medical-imaging |
| - token-pruning |
| - self-supervised-learning |
| - label-free |
| - vision-transformer |
| - negative-results |
| library_name: pytorch |
| --- |
| |
| # covtoken — Label-Free Lesion-Subspace Token Economy for Medical Imaging |
|
|
| Code, gated evaluation, and a working paper draft for a label-free token-pruning method on |
| frozen self-supervised medical vision transformers. |
|
|
| > **Reframed contribution (see `gate_reports/SUMMARY.md`).** The load-bearing result is a |
| > **label-free lesion subspace** — a mid-layer geometry that localizes lesions WITHOUT labels |
| > across anatomy, modality (CT + ultrasound), and backbone (MedDINOv3 + DINOv2) — together with |
| > **membership pruning** (beats saliency pruning on small-lesion miss-rate), a **conformal |
| > retention certificate**, and **lesion-routed depth** (1.6× FLOPs). The original |
| > *coverage-constrained optimization with an interpretable dual* is reported as a **clean |
| > negative result** with a transferable mechanism (rank-based coverage rewards diverse spanning, |
| > rare pathology needs concentration). See `gate_reports/NEGATIVE_RESULT.md`. |
| |
| ## Headline numbers |
| |
| | | | |
| |---|---| |
| | Mid-layer finding | lesion AUROC final-layer 0.565 → block-3 **0.871** | |
| | Cross-modality | density-A: lung CT 0.87, kidney 0.82, **breast US 0.73** (DINOv2; attention 0.49) | |
| | Membership > saliency pruning | LIDC +27.6/+15.8, KiTS23 +7.4, BUSI +13.8/+19.0 pts | |
| | Conformal retention cert. | empirical coverage 0.978 ≥ nominal 0.90 | |
| | Lesion-routed depth | 1.6× FLOPs at 98% small-lesion sensitivity | |
| | Negative result | coverage floor 0.22 vs membership 0.82 (small-lesion recall) | |
| |
| ## Layout |
| |
| ``` |
| subspace/ label-free lesion subspace: density (A) + residual (B) constructions |
| coverage/ rankme / coding-rate / energy functionals (the FALSIFIED coverage objective) |
| gate/ constrained pruner (Gumbel mask + dual) + per-image certificate [negative result] |
| arch/ conformal_head, routed_depth, volumetric (Phase-6 components) |
| backbone/ frozen MedDINOv3 ViT-B/16 loader (DINOv2 used for ultrasound, in jobs/) |
| data/ CT token-bank builder, eval-only mask loading + label-leak guard |
| eval/ DeLong / paired-bootstrap / Spearman stats; gate runners |
| jobs/ all experiments as Hugging Face Jobs (materialize, banks, gates, ablations) |
| gate_reports/ machine-readable per-gate decision records + SUMMARY + NEGATIVE_RESULT |
| configs/ thresholds.lock.json (Phase-1b calibrated, locked) |
| paper/ working_draft.md + figures/ + venue_notes.md |
| tests/ CI label-leak test (no label may touch subspace construction) |
| ``` |
| |
| ## Reproducibility |
| |
| All experiments ran as Hugging Face Jobs. Backbones: `ricklisz123/MedDINOv3-ViTB-16-CT-3M` (CT), |
| `facebook/dinov2-base` (ultrasound). Datasets: LIDC-IDRI, KiTS23, MSD Task03 Liver, MSD Task07 |
| Pancreas, BUSI. Masks are evaluation-only; `tests/test_label_leak.py` fails the build if a label |
| reaches subspace construction. The frozen DINOv3 model code is installed from |
| `github.com/facebookresearch/dinov3` at job time (not vendored here). |
| |
| Decision records: `gate_reports/`. Locked thresholds: `configs/thresholds.lock.json`. |
| |
