task47_0809/task8_1000samples_run/setup.md

Task 8 — I-JEPA context-vs-EMA-target run (official ImageNet-1K, 1,000 samples)

What's in this package

• scripts/ — the Task 8 evaluator (task8_ijepa_ema_ambiguity.py) plus its supporting scripts (checkpoint downloader, official ImageNet exporter, aggregate-plot helper).
• results/ — the complete output of this run only: task8_metrics.json, context_target_layer_metrics.csv, per_class_layer_metrics.csv, per_sample_layer_metrics.csv, the CKA-depth and KL-ambiguity-variance PDFs, and a reproducibility/ bundle (exact run command, dataset manifest, and a source snapshot of the script as actually run).
• imagenet_1000_samples/ — the exact 1,000 images used, one folder per class (50 classes × 20 images). Same 1,000 images used for this session's Task 4 and Task 7 runs too.

Where the data comes from

The 1,000 images are drawn from the official ILSVRC/imagenet-1k validation split (Hugging Face, gated/authenticated access — not a public mirror), sampled deterministically with seed 42. See results/reproducibility/setup.md for the exact manifest SHA-256 and per-run provenance notes.

Requirements

• A CUDA-enabled PyTorch environment with torchvision installed.
• The WorldModelLens repository checked out, with world_model_lens importable (editable install or PYTHONPATH) — these scripts import it and are not self-contained.
• The official I-JEPA ViT-H/14 ImageNet-1K checkpoint (vith14_in1k_ep300.pth.tar) — download with scripts/download_ijepa_weights.py --dest checkpoints --verify if you don't already have it (~10.4 GB).
• A frozen torchvision ViT-H/14 baseline — downloaded automatically on first run (IMAGENET1K_SWAG_LINEAR_V1, ~2.35 GB) unless you pass --baseline-checkpoint with a local state dict.
• An inpainting bank for the ambiguity-evaluation part of the script (required argument). This run used a synthetic smoke-test bank (data/task8_official_imagenet_synthetic_bank in the source repo) — its results are pipeline artifacts, not scientific inpainting results. Any bank matching that folder layout (case_NNN/{source.png, candidate_0..2.png, metadata.json}) will work.

Run

From the WorldModelLens repo root, with this package's imagenet_1000_samples/ copied somewhere accessible and the checkpoint in place:

python scripts/task8_ijepa_ema_ambiguity.py \
  --ijepa-checkpoint checkpoints/vith14_in1k_ep300.pth.tar \
  --imagenet-val /path/to/imagenet_1000_samples \
  --image-manifest results/reproducibility/dataset_manifest.json \
  --inpainting-bank /path/to/an/inpainting_bank \
  --n-validation 1000 --batch-size 8 --seed 42 --cka-depths all \
  --device cuda --output-dir results

The --image-manifest flag makes the script use exactly the 1,000 bundled images in their original order — --imagenet-val is still required by the script but only matters if you omit --image-manifest. The exact command used to produce the results in this package is in results/reproducibility/run_command.txt.

Headline results from this run

• Context encoder and EMA target encoder are nearly identical (CKA > 0.99) through layer ~20, then diverge sharply from layer 24 onward (CKA down to 0.56 at layer 31), with a partial rebound at the final layer (0.94 at L32).
• I-JEPA's candidate-ambiguity entropy is much lower (more confident/peaked) than the torchvision baseline's on all 3 synthetic smoke-test cases, with large KL divergence between the two models' candidate distributions (114–615) — see results/task8_metrics.json for exact per-case numbers.