agu18dec/olens-ablations

OLens ablations for Qwen3.6-27B

Controlled ablations of the OLens recipe published at agu18dec/final_olens. Every checkpoint here loads with the same self-contained loader (olens.py, included), and each study changes exactly one thing against a matched control.

What an OLens is

An OLens reads a residual-stream vector of Qwen3.6-27B (any of its 64 layers) and writes what it encodes as text. It is trained in three stages, all LoRA on the frozen 27B:

1. AR (reconstructor): reads [Layer NN] + a span of text (1–32 tokens) and predicts the direction of the residual at the token just before that span. Loss 2(1 − cos) on unit vectors, raw (not whitened) activations.
2. Inverter: the reverse map, trained by cross-entropy to reproduce a text span from the AR's predicted vector for it, injected as a unit vector (×1000) at a slot token of a fixed prompt.
3. RL: GRPO on true residuals (64 readouts per activation, 8 activations per step, KL .005 to the inverter, lr 5e-5). Reward: the frozen AR reads the readout; the score is the centred fraction of variance explained, (cos² − ‖μ‖²)/(1 − ‖μ‖²), where ‖μ‖² is the share of the layer's mean direction.

Quickstart

from olens import from_hub, load_lens, residual, read
lens = load_lens(from_hub("agu18dec/olens-ablations", "training_text/web_text/lens_rl_step150"))
h = residual(lens, "Who wrote the opera Carmen?", layer=40)
print(read(lens, h, layer=40, n=4))

Each folder holds model.safetensors (trainable weights only; the 27B is loaded from the Hub) and export.json (the exact training configuration). Studies without their own ar/ were trained with the reference AR in agu18dec/final_olens/ar.

How results are measured

WorkspaceBench headline (mean over 23 families, all-Opus judge, one readout per cell, the benchmark's own sampling), with hallucination as 1 − hallucination rate (higher = fewer false claims). One run per arm; run-to-run SD of the headline ≈ .015, item-bootstrap 95% CI ≈ ±.016. * = mean over the families the judge could complete (n/24), when it refused some cells. The reference lens scores 0.663 at RL step 150 and 0.683 at step 600 (agu18dec/final_olens).

1. What text the AR is trained on — training_text/

Four ARs, each trained on 13.58M examples of a different text source (otherwise identical), then the same inverter and RL recipe on top of each.

folder	AR training text	AR held-out loss (own data)	inverter	RL step 150	hallucination
chat_own_replies	Qwen3.6-27B's own replies to WildChat turns (the reference recipe's data)	0.283	0.594	0.686	0.67
chat_other_model_replies	the original assistant replies stored in WildChat (written by other models)	0.329	0.588	0.691	0.62
web_text	FineWeb web text	0.374	0.592	0.683	0.60
web_text_own_continuations	Qwen3.6-27B's own continuations of FineWeb prefixes	0.367	0.590	0.679	0.64

Finding: the text source changes how well the AR fits (own chat replies are easiest) but not the final lens: after RL all four are within noise.

2. Which residual the AR predicts — ar_target/

Five ARs (4.8M examples each, the same rows) trained to predict different targets; the lens is always used on the residual at the token before the span, so RL scores every arm against that.

folder	AR target	AR loss (own target)	inverter	RL step 150	hallucination
next_token	the residual at the token just before the span (the reference recipe)	0.318	0.562	0.613	0.66
mean_last_4	the mean of the residuals over the last 4 tokens before the span	0.224	0.561	0.603	0.69
mean_last_16	the mean over the last 16 tokens	0.154	0.543	0.612	0.69
offset_4	the residual 4 tokens before the span	0.370	0.551	0.592	0.66
offset_16	the residual 16 tokens before the span	0.396	0.528	0.593	0.69

Finding: predicting the next-token residual is best or tied; averaging targets is easier to fit (lower loss) but does not help the lens, and offsets hurt.

3. Where RL starts — rl_starting_point/

RL with the reference AR and reward from different starting points.

folder	start	before RL	RL step 150	RL step 300 / 600	hallucination (last)
from_base_model	the base model (no inverter)	–	0.013	0.013 (600)	–
from_base_model_nla_log_reward	the base model, NLA's log reward −log 2(1 − c)	–	0.013	0.013 (300)	–
from_base_model_centred_reward	the base model, reward computed after subtracting the layer mean from both vectors	–	0.573* (19/24)	0.582* (22/24) (300)	0.71
from_inverter_step123	the inverter after 123 of its 1,172 steps	0.500	0.601	0.642 (300)	0.63
from_inverter_step489	the inverter after 489 steps	0.580	0.647	0.657 (300)	0.67
(reference)	the full inverter	0.594	0.663	0.673 (300)	0.68

Finding: from the base model RL collapses: every readout becomes the single token <. Content- free text makes the AR predict the layer's mean direction, which already matches any activation (cos² ≈ .64), while the base model's off-topic readouts score below that, so RL learns to say nothing. Subtracting the layer mean inside the reward removes that free score and RL learns again (with messy readouts). NLA's log reward collapses the same way: every raw-cosine reward has this floor. Starting from the inverter avoids the trap; a longer inverter still helps.

4. Reward shape — reward_shape/

The reference lens's RL with the reward replaced; everything else identical. c = cosine between the AR's prediction and the true residual (raw vectors).

folder	reward	RL 150	RL 300	RL 480	RL 600
(reference)	(c² − ‖μ‖²)/(1 − ‖μ‖²)	0.663	0.673	0.676	0.683
nla_log_reward	−log 2(1 − c)	0.663	–	–	–
nla_exact_reward	−2(1 − c)	0.678	0.675	0.654	0.639

Finding: all three rank a group's readouts the same way and tie early; −2(1 − c) declines after step 300.

5. How long to run RL — rl_length/

The reference RL continued from step 600 at the same constant learning rate.

RL step	150	300	600	960	1,200
WorkspaceBench	0.663	0.673	0.683	0.661	0.678
hallucination	0.66	0.68	0.62	0.65	0.60

Finding: the headline plateaus around step 600 while hallucination gets worse.

6. RL variants — rl_co_trained_ar/, rl_advantage_normalisation/

The reference RL with one change each.

• Co-trained AR: the AR keeps training during RL (as NLA's critic does): after every step it takes one AdamW step (lr 8e-5) on the best readout of each group, loss 2(1 − cos) to the true activation. Only the lens is published here.
• Advantage normalisation: rewards centred per group, then divided by one standard deviation pooled over the whole step (groups with zero spread get no update), instead of each group's own.

folder	RL 150	RL 300	RL 600	hallucination 150 / 300 / 600
(reference)	0.663	0.673	0.683	0.66 / 0.68 / 0.62
rl_co_trained_ar	0.696	0.717	0.694* (23/24)	0.60 / 0.56 / 0.54
rl_advantage_normalisation	0.675	0.678	0.676	0.67 / 0.69 / 0.69

Finding: co-training raises WorkspaceBench early but makes readouts about twice as long and steadily less accurate (more invented claims); the co-trained AR scores its own lens higher and higher while the original AR does not. Pooled advantage normalisation changes nothing.

7. Bullet readouts — bullets/

The lens is distilled into readouts of up to 4 - bullets, then RL scores the bullets jointly (best non-negative combination of their AR directions).

• self-chosen: for each activation the inverter samples 16 readouts, the AR picks the line prefixes that together best rebuild the activation (greedy non-negative least squares); the student starts from the inverter.
• dictionary: bullets are picked the same way from a fixed dictionary of real text phrases; the student starts from the base model.

folder	student (before RL)	RL 150	RL 300	RL 150, 4 readouts	hallucination (RL 150)
self_chosen_bullets	0.620	0.761	0.746	0.805	0.53
dictionary_bullets	0.286	0.652	0.676	0.731	0.56

Finding: self-chosen bullets are the strongest single-read lens we have (also published as agu18dec/final_olens/p3_bullets_rl_step150); dictionary bullets are far weaker. Both state more false things than the single-readout lens.

8. Gradient instead of activation — gradient_target/

Same rows, AR recipe and downstream recipe as ar_target/next_token, but the AR predicts g = ∂/∂h Σ log p(span tokens): the gradient, with respect to the activation h at the token before the span, of the log-likelihood of the span the AR reads (computed exactly through Qwen3.6-27B with the full preceding context). The lens is RL'd and read on true gradients; at read time the span is the next 16 tokens of text (extended by a sampled continuation when the text ends). olens.py captures activations only: to read a gradient, compute g as defined and pass it to read.

folder	what it reads	inverter	RL 150	RL 300	hallucination (RL 300)
gradient_target	gradients	0.108	0.138	0.146	0.90
gradient_target/inverter_rl_on_activations	activations (gradient-trained inverter, RL on activations)	–	0.579	0.620	0.69
ar_target/next_token (control)	activations	0.562	0.613	0.622	0.65

Finding: gradients are much harder to predict from text than activations (the AR explains ~3% of the gradient's direction beyond the mean baseline vs ~21% for the activation), and a lens that reads gradients writes mostly generic text. The gradient-trained inverter is still a fine starting point for reading activations.

Files

olens.py                        self-contained loader: from_hub, load_lens, load_ar, residual, read, ar_direction
training_text/<arm>/{ar,inverter,lens_rl_step150}/
ar_target/<arm>/{ar,inverter,lens_rl_step150}/
rl_starting_point/<arm>/{inverter,lens_rl_step*}/
reward_shape/<arm>/lens_rl_step*/
rl_length/lens_rl_step{960,1200}/
rl_co_trained_ar/lens_rl_step{150,300,600}/
rl_advantage_normalisation/lens_rl_step{150,300,600}/
bullets/{self_chosen_bullets,dictionary_bullets}/{student_sft,lens_rl_step150,lens_rl_step300}/
gradient_target/{ar,inverter,lens_rl_step150,lens_rl_step300}/, inverter_rl_on_activations/