---
license: mit
tags:
  - medical-imaging
  - token-pruning
  - self-supervised-learning
  - label-free
  - vision-transformer
  - negative-results
library_name: pytorch
---

# covtoken — Label-Free Lesion-Subspace Token Economy for Medical Imaging

Code, gated evaluation, and a working paper draft for a label-free token-pruning method on
frozen self-supervised medical vision transformers.

> **Reframed contribution (see `gate_reports/SUMMARY.md`).** The load-bearing result is a
> **label-free lesion subspace** — a mid-layer geometry that localizes lesions WITHOUT labels
> across anatomy, modality (CT + ultrasound), and backbone (MedDINOv3 + DINOv2) — together with
> **membership pruning** (beats saliency pruning on small-lesion miss-rate), a **conformal
> retention certificate**, and **lesion-routed depth** (1.6× FLOPs). The original
> *coverage-constrained optimization with an interpretable dual* is reported as a **clean
> negative result** with a transferable mechanism (rank-based coverage rewards diverse spanning,
> rare pathology needs concentration). See `gate_reports/NEGATIVE_RESULT.md`.

## Headline numbers

| | |
|---|---|
| Mid-layer finding | lesion AUROC final-layer 0.565 → block-3 **0.871** |
| Cross-modality | density-A: lung CT 0.87, kidney 0.82, **breast US 0.73** (DINOv2; attention 0.49) |
| Membership > saliency pruning | LIDC +27.6/+15.8, KiTS23 +7.4, BUSI +13.8/+19.0 pts |
| Conformal retention cert. | empirical coverage 0.978 ≥ nominal 0.90 |
| Lesion-routed depth | 1.6× FLOPs at 98% small-lesion sensitivity |
| Negative result | coverage floor 0.22 vs membership 0.82 (small-lesion recall) |

## Layout

```
subspace/     label-free lesion subspace: density (A) + residual (B) constructions
coverage/     rankme / coding-rate / energy functionals (the FALSIFIED coverage objective)
gate/         constrained pruner (Gumbel mask + dual) + per-image certificate  [negative result]
arch/         conformal_head, routed_depth, volumetric  (Phase-6 components)
backbone/     frozen MedDINOv3 ViT-B/16 loader (DINOv2 used for ultrasound, in jobs/)
data/         CT token-bank builder, eval-only mask loading + label-leak guard
eval/         DeLong / paired-bootstrap / Spearman stats; gate runners
jobs/         all experiments as Hugging Face Jobs (materialize, banks, gates, ablations)
gate_reports/ machine-readable per-gate decision records + SUMMARY + NEGATIVE_RESULT
configs/      thresholds.lock.json (Phase-1b calibrated, locked)
paper/        working_draft.md + figures/ + venue_notes.md
tests/        CI label-leak test (no label may touch subspace construction)
```

## Reproducibility

All experiments ran as Hugging Face Jobs. Backbones: `ricklisz123/MedDINOv3-ViTB-16-CT-3M` (CT),
`facebook/dinov2-base` (ultrasound). Datasets: LIDC-IDRI, KiTS23, MSD Task03 Liver, MSD Task07
Pancreas, BUSI. Masks are evaluation-only; `tests/test_label_leak.py` fails the build if a label
reaches subspace construction. The frozen DINOv3 model code is installed from
`github.com/facebookresearch/dinov3` at job time (not vendored here).

Decision records: `gate_reports/`. Locked thresholds: `configs/thresholds.lock.json`.