oracle-lens-ao-checkpoints — Inverted OLens AO (activation oracles)
Checkpoints only. All data lives in
agu18dec/oracle-lens-data
(ao/pool/ crop pools, ao/arout/ the AR reconstructions these AOs train on, ao/runs/ the
frozen injection scale); the AR checkpoints these are built on in
agu18dec/oracle-lens-ar-checkpoints.
Training runbook: docs/project/experiments/ola/iolens_runbook.md in the
global-workspace repo.
An AO is the inverse of the AR: it takes ONE reconstructed residual-stream vector — scaled raw,
no whitening — injected as a soft token (the placeholder slot of a 55-token "explain" prompt,
forward via inputs_embeds) and is trained with CE to emit the span text
<explanation>\n{span}\n</explanation>.
Runs
| folder | built on | config | status |
|---|---|---|---|
ao.iolens.chat.k4.L20plus.s0/ |
chat AR FINAL ex16014240 |
k=4 layers/crop, layers 20–63 only, alpha 16000 (scale 64.559), lr 3e-4, eff. batch 768×6-GPU | segment 1 (pool_iolens, 527k crops) |
ao.iolens.chat.k4.L20plus.s1/ |
warm start from s0's last step | same, trained on pool_iolens_ext1 (939k crops), validated on s0's exact val set |
extension segment (when present) |
Each stepN/ is a full adapter checkpoint; validation runs every 200 steps
(val_ce, per-layer val_ce_L{20..63}, per-span-length val_ce_N{2..64}). The s0→s1 val-CE
curve is directly comparable: the extension segment validates on the parent pool's
conversation-split val set (same split/layer seeds ⇒ identical examples).
Two things a consumer must know
- The arout shards are self-describing (
ao_layers,ar_layers,n_universe, pick seeds in metadata) — the layer a row corresponds to comes from there, nowhere else. - The injection scale is frozen (
ao/runs/scale_iolens_chat_final.json); val CE was insensitive across 64× of alpha (LayerNorm absorbs magnitude), so do not re-fit it per run.