position vs FINO (Meta FAIR, arXiv:2606.05107): orthogonal frozen-probe vs metadata adaptation; de-risks F1b; complementary experiment noted
Browse files- RESEARCH_OVERVIEW.md +17 -0
- paper/working_draft.md +12 -0
RESEARCH_OVERVIEW.md
CHANGED
|
@@ -50,6 +50,23 @@ preserving pretraining via reconstruction) returned **negatives that sharpened t
|
|
| 50 |
than producing new methods — reported as first-class results. The remaining novel lever is G2
|
| 51 |
(cross-domain universality), a deliberate new project, not a quick eval.
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
## Reproduce
|
| 54 |
All jobs in `jobs/` (PEP-723 uv scripts) run via `hf jobs uv run --flavor <t4-medium|t4-small|cpu>
|
| 55 |
--secrets HF_TOKEN -v hf://buckets/Chucks90/eryon-datasets:/mnt <script>`. Specs in
|
|
|
|
| 50 |
than producing new methods — reported as first-class results. The remaining novel lever is G2
|
| 51 |
(cross-domain universality), a deliberate new project, not a quick eval.
|
| 52 |
|
| 53 |
+
## Concurrent work & positioning
|
| 54 |
+
**FINO** (Gardès et al., *Who Needs Labels? Adapting Vision Foundation Models With the Metadata You
|
| 55 |
+
Already Have*, arXiv:2606.05107, Meta FAIR, June 2026; code on the DINOv3 FINO branch) adapts vision
|
| 56 |
+
FMs to scientific domains label-free via **metadata-guided SSL training**. It is **orthogonal**:
|
| 57 |
+
FINO *trains/adapts* a backbone with metadata; covtoken *probes a frozen* backbone with pure token
|
| 58 |
+
geometry — no training, no metadata. It addresses none of covtoken's contributions (depth/layer
|
| 59 |
+
localization, the rank-vs-concentration law, the conformal certificate, rare-structure focus).
|
| 60 |
+
Three implications:
|
| 61 |
+
- **Differentiate** in related work: frozen geometric probe + law + certificate, not adaptation.
|
| 62 |
+
- **De-risks F1b**: G1a showed the eroded subspace cannot be recovered *training-free* (globalization
|
| 63 |
+
is entangling). FINO is an existence proof that *with* a training signal one can "preserve
|
| 64 |
+
informative, suppress spurious" factors — so the practical concentration-preserving lever is
|
| 65 |
+
**metadata-guided adaptation**, not steering.
|
| 66 |
+
- **Complementary experiment** (new future direction): run the covtoken probe on a FINO-adapted
|
| 67 |
+
backbone — does metadata-guided adaptation preserve the mid-layer concentration subspace at depth
|
| 68 |
+
and raise rare-signal separability? Bridges the two lines and rides their release.
|
| 69 |
+
|
| 70 |
## Reproduce
|
| 71 |
All jobs in `jobs/` (PEP-723 uv scripts) run via `hf jobs uv run --flavor <t4-medium|t4-small|cpu>
|
| 72 |
--secrets HF_TOKEN -v hf://buckets/Chucks90/eryon-datasets:/mnt <script>`. Specs in
|
paper/working_draft.md
CHANGED
|
@@ -205,6 +205,18 @@ mechanistic negative result on rank-based coverage objectives. Note (iv) gives c
|
|
| 205 |
from a companion representation-coverage probe study: if such a probe reads final-layer features,
|
| 206 |
Finding 1 says it reads the wrong layer — the two results reinforce rather than overlap.
|
| 207 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 208 |
## 8. Limitations
|
| 209 |
|
| 210 |
- The method helps only where feature density localizes the lesion (liver = characterized
|
|
|
|
| 205 |
from a companion representation-coverage probe study: if such a probe reads final-layer features,
|
| 206 |
Finding 1 says it reads the wrong layer — the two results reinforce rather than overlap.
|
| 207 |
|
| 208 |
+
**Relation to label-free FM adaptation (FINO; Gardès et al., 2026).** Concurrent work adapts vision
|
| 209 |
+
foundation models to scientific domains *without task labels* by guiding a self-supervised objective
|
| 210 |
+
with **metadata**, training the backbone. Our work is orthogonal and complementary: we keep the
|
| 211 |
+
backbone **frozen**, use **no metadata or labels**, and contribute a geometric *analysis* (where the
|
| 212 |
+
signal lives), a token *economy* (membership pruning, routed depth), a retention *certificate*, and
|
| 213 |
+
a *law* on objective choice — none of which an adaptation method addresses. The two compose: our
|
| 214 |
+
probe can be run on a FINO-adapted backbone to test whether metadata-guided adaptation preserves the
|
| 215 |
+
mid-layer concentration subspace at depth and improves rare-signal separability (a question our
|
| 216 |
+
training-free steering result, Sec. 6 / companion study, shows cannot be solved without a training
|
| 217 |
+
signal). Their result is also indirect support for our mechanism: counteracting depth-globalization
|
| 218 |
+
of informative local factors is plausibly part of why metadata guidance helps.
|
| 219 |
+
|
| 220 |
## 8. Limitations
|
| 221 |
|
| 222 |
- The method helps only where feature density localizes the lesion (liver = characterized
|