oracle-lens-ar-checkpoints — Inverted OLens AR (activation reconstructors)
Checkpoints only. All data (rollouts, whiteners, AO pools/arout, provenance) lives in
agu18dec/oracle-lens-data;
AO checkpoints in
agu18dec/oracle-lens-ao-checkpoints.
Training runbook: docs/project/experiments/ola/iolens_runbook.md in the
global-workspace repo.
An AR maps a text span to the residual stream Qwen3.6-27B was in immediately before emitting it
(span_start − 1), at 16 layers (4, 8, …, 60, 63) at once: truncated 27B backbone + LoRA r16
- one shared LN/Linear head + a layer embedding. Trained with per-layer whitened cosine loss.
Runs
| folder | cell | status | val FVE mean / band L20–56 / L63 |
|---|---|---|---|
ar.chat.mlayer.lc.s0/ |
assistant rollouts | FINAL at ex16014240 (tag chat-AR-final-ex16014240) |
14.37% / 15.36% / 28.20% |
ar.pt.mlayer.lc.s0/ |
pretraining-text rollouts | stopped (resumable), last rung ex16013824 |
8.32% / — / 16.14% |
Every subfolder ex<examples>/ is one milestone rung: {lora/, heads.pt, meta.json}, where
meta.json carries the exact all-reduced examples and tokens_span plus every val metric
— the rung series of one constant-LR run is the scaling curve (plots in curves/, regenerable
with scripts/ola/iolens_plot_ar_curve.py). Chat FINAL details: its
FINAL.md.
Loading
from global_workspace.ola.ar_loader import fetch_ar_checkpoint, load_ar
path = fetch_ar_checkpoint("ar.chat.mlayer.lc.s0/ex16014240") # or a local $OLA_ROOT path
The trained layer set is derived from heads.pt (layer_emb.weight.shape[0] = 16 here — layer 0
was dropped). Never assume 17 layers; every consumer that hardcoded len(LAYERS) has broken.
FVE numbers are in the whitened unit-norm basis (whiteners in the data repo); a random vector
scores ~1/5120 in that basis.