Centered Ablation: Why Last-Layer Directional Ablation Under-Fires or Collapses, and a Closed-Form Fix
Abstract
Directional ablation removes a hidden state's component along a "behaviour direction" while a language model runs. It is the standard way to switch a behaviour such as refusal off, but it is erratic: on some models it barely fires, on others the text collapses into repetition. We show why, and fix it. At the last residual stream, ablation is exactly a logit bias: a fixed vocabulary footprint scaled by a gate, the cosine between the hidden state and the direction. The gate is measured from the origin, but last-layer states share a large common mode far from it, so a direction that overlaps the common mode fires on benign text and under-fires on its targets. Centered Ablation (CA) projects the common mode out of the direction, in closed form. Across 8 models, CA raises attack success by 29 and 46 points where the standard recipe under-fires, cuts collapse on Gemma-2-9B from 50\% to 1\%, and preserves 12 to 40 points more benign answers on high-overlap models, 39 to 46 more than the affine offset of prior work. Two numbers computed before generation predict, on models the analysis had not seen, which gain faithfulness and which gain attack success, and a collapse index marks when text degenerates. We also show that two standard safety judges score collapsed text in opposite directions, which can fabricate a 46-point effect.
Community
At the last layer, directional ablation is exactly a logit bias scaled by a cosine with the hidden state. Hidden states share a large common mode far from the origin, so the standard direction fires on benign text and under-fires on its target. Centered Ablation projects the common mode out of the direction: closed form, no new hyperparameter.
Been fighting exactly this: a refusal direction I pulled from one checkpoint transfers as a total no-op, while the identical recipe on a sibling model turns every output into repetition. If the gate really is a from-origin cosine against a mean state parked way out on some shared direction, then what I've been calling "this vector works" was never a claim about refusal — it was a claim about that checkpoint's common mode. The test I'd run: measure cosine between each candidate direction and the per-layer mean hidden state before trusting it, then re-measure after every fine-tune, since the offset drifts and my steering configs silently go stale. Curious whether the closed-form centering also fixes the cases where ablation "worked" but only at scale factors I'd never ship — if it does, half my alpha sweeps were just compensating for a bad baseline rather than finding the real gain.
Thanks, that matches our view, with one scope note: the result is exact only at the last residual stream, where ablation is a logit bias scaled by the from-origin cosine. We make no claim about earlier layers.
Your pre-flight test is essentially our overlap ω: the cosine between the direction and the mean of RMS-normalized benign states. High ω means the edit fires on ordinary text. Two more pre-generation numbers separate your symptoms: efficacy η (low means your "no-op") and a collapse index β (≳ 1 means your "repetition"). We didn't study fine-tune drift, but ω is two means, so re-measuring it per checkpoint is cheap.
On alpha sweeps, partly yes. A large α makes up for a small target gate but scales the benign bias by the same factor. At s = 1, centering takes Llama-3.2-3B from 10% to 57% attack success and Gemma-2-9B from 50% to 1% collapse. But on Gemma-2-9B, DIM at s = 0.5 is also collapse-free, and on six of our eight models CA changes little. It helps when ω is high or η is low, not everywhere.
Get this paper in your agent:
hf papers read 2611.20148 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper