--- license: mit tags: - learnability-prediction - training-free - zero-cost-proxy - edge-vision - tiny-models - reproducibility language: - en pretty_name: "LogLens — Training-Free Task-Learnability Prediction for Nano Vision Models" --- # LogLens: can you tell whether a tiny vision model will learn a task — before training it? **One number, computed from task imagery alone (no network, no gradients, no training), predicts whether a ~10⁵-parameter model can learn a visual change-dating task. Blind-validated at Spearman ρ = 0.988 on ten pre-registered conditions.** This repo contains the predictor, the mechanism that explains why it works, the measured boundary where it *stops* working, and a one-command harness so you can reproduce — or kill — the result on your own machine and your own scene. ## Why this might interest you - **It is task-side, not architecture-side.** Zero-cost NAS proxies (synflow, GradNorm, ZiCo, NTK scores) rank *architectures* for a fixed dataset using the network's gradients at init. This asks the complementary question: *given a fixed small model, is this task learnable at all?* The pixel-space predictor uses **no network whatsoever**. - **There is a mechanism, not just a correlation.** The predictor works because an *untrained* encoder preserves and amplifies the pixel signal-to-clutter ratio into the feature space its head learns from (ρ = 0.981 across 28 task cells; ρ = 0.932 against trained accuracy, 5 seeds per config). - **The failure boundary is measured, not assumed.** On natural-image classification the predictor *inverts* (ρ = −1.00) and **no** trainless variant rescues it. We explain why, and state the scope precisely. - **Everything was pre-registered, and the failures are published.** Five claims were falsified by their own controls during this work; all are reported. ## Quickstart — reproduce (or refute) it in one command ```bash pip install torch numpy opencv-python scipy python replicate.py --bank --full --seeds 3 ``` It builds a 64px bank from your photos, computes the predictor, trains a size grid blind, and prints a PASS/FAIL scorecard: | test | bar | our result | |---|---|---| | P1 — predictor ↔ feature-space predictor | ρ ≥ 0.90 | 0.981 (home), 0.969 (2nd scene) | | P2 — **the law**: predictor ↔ learnability | ρ ≥ 0.85 | 0.988 (10-cell blind); 0.932 / 0.850 at 5 seeds | | P3 — ignition present (learner escapes, floor pinned at log K) | present | CNN + ViT, both scenes | Compare against `EXPECTED_RESULTS.md`. **Disagreement is a result** — please open a discussion with your `results_replicate.json` and scene hash. ## Read this before you run it (hard-won) 1. **The learn/fail boundary is seed-bimodal and slow.** A cell can look "collapsed" at 8k steps and escape at 18k. Escape times spread 6k–18k. Always use ≥ 2 seeds (ideally 3) and `--full`. Single-seed results near the boundary are noise. 2. **Collapse = loss ≈ log(num_classes)** exactly (uniform guessing). Use it to detect dead runs with zero eval cost. 3. **Compare *ordering* across scenes, never absolute thresholds.** Ordering is scene-universal (ρ = 0.996 across two scenes); the threshold is scene-local, so a cell that floors on our scene may learn on yours. That is expected. 4. **Don't use the raw signal/clutter *ratio* to judge escape mid-training** — it is confounded (it spikes when the encoder degenerates and clutter → 0). Track the *signal* component or the head's margin. ## What's here ``` replicate.py one-command harness (PASS/FAIL scorecard) EXPECTED_RESULTS.md our numbers + the caveats above REPLICATION_KIT.md long-form protocol (Windows OK, CPU OK) DRAFT_paper_v3.md the paper draft LITERATURE_CHECK.md what is already published (and what we therefore do NOT claim) triclock/, simreal/ the ~128K-param model + the task generator (imported by replicate.py) code/compute_R.py the pixel-space predictor, standalone code/compute_R_grid.py predictor over a size × contrast grid code/compute_featR.py the same quantity through an untrained encoder (the mechanism) code/compiler_v2.py automated measure → route → escape-and-select results/ the JSONs behind every number in the paper ``` ## Scope — please read before generalizing This **works** where the discriminative signal already lives in a fixed representation: our change-dating task (ρ = 0.988) and algorithmic phase-transition tasks. It **provably fails** for natural-image classification, where within-class variance *is* the learnable material — we tested pooled, high-pass, edge, random-conv, gated, and k-NN forms; all fail (best ρ ≈ +0.04 against a ground-truth ordering, with the plain predictor at ρ = −1.00). Grokking results in the paper's §7 are **reported as supporting evidence only**; that ground is already covered by OmniGrok (ICLR 2023) and 2026 work on norm-separation delay laws and interventional/predictive grokking frameworks. We claim no novelty there. See `LITERATURE_CHECK.md`. ## Limitations One scene family, one architecture class, one training recipe, n = 10 blind conditions, and **no external replication yet**. That last one is the point of this release. ## Citation ```bibtex @misc{loglens2026, title = {Predicting Task Learnability for Nano Vision Models --- and Why the Prediction Works}, author = {Cox, Joe and {Claude (Anthropic)}}, year = {2026}, note = {Draft. Code and pre-registrations: this repository.} } ``` Experiments were run on a single Jetson AGX Orin. Every prediction in the paper was registered to a timestamped file before the corresponding run.