File size: 3,492 Bytes
286be0a 46a6bed | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
base_model:
- Qwen/Qwen3.6-27B
- Qwen/Qwen2.5-7B-Instruct
tags:
- persona-vectors
- steering-vectors
- activation-steering
- interpretability
---
## Layout
```
qwen3.6-27b/ directions extracted on Qwen/Qwen3.6-27B (hidden 5120)
qwen2.5-7b/ directions extracted on Qwen/Qwen2.5-7B-Instruct (hidden 3584)
directions.npz 35 traits x 64 layers
config.json extraction config (battery, read position, seed, git hash)
summary.json run summary (n samples, geometry layer, placebo gate)
geometry.json pairwise geometry of the direction set
survival.json which traits survive the permutation test
battery_report.jsonl per-trait selected layer, held-out AUROC, gap, Cohen's d
```
The vectors are fit **separately on each base model**, so a direction from one model
is meaningless applied to the other, and readings are only comparable within a model.
## `directions.npz` schema
Keys are `"<trait>|<layer>|<kind>"`:
- `unit` — the L2-normalized direction (use this to project or steer)
- `gap` — the un-normalized difference of means (its norm is the raw class gap)
- `split_half` — the same direction fit on a held-out half, for reliability
Layers run `1..64`. The extra key `n_slices` (= 65) is the layer count including the
embedding slice at index 0. Each vector has length = the model's hidden size.
```python
import numpy as np
from huggingface_hub import hf_hub_download
f = np.load(hf_hub_download("knowingly-wrong/persona-vectors",
"qwen3.6-27b/directions.npz"))
v = f["dishonesty|16|unit"] # (5120,)
score = activations @ v # project residual-stream activations onto the trait
```
## The 35 traits, and the 8 the paper reads
All 35 traits are extracted and shipped. The paper reports only these 8 (each oriented
so that a **positive** projection means **more of the undesirable trait**):
evil, dishonesty, hallucination, overconfidence, sycophancy, manipulativeness,
machiavellianism, callousness.
Each trait is read at one selected layer (chosen on held-out data, in
`battery_report.jsonl`). The selected layer and Cohen's d differ per model:
| trait | 27B layer | 27B d | 7B layer | 7B d |
|---|---|---|---|---|
| evil | 35 | 0.72 | 15 | 0.78 |
| dishonesty | 16 | 4.16 | 26 | 1.09 |
| hallucination | 34 | 2.50 | 21 | 0.59 |
| overconfidence | 45 | 3.90 | 21 | 1.73 |
| sycophancy | 64 | 6.31 | 27 | 1.62 |
| manipulativeness | 51 | 2.78 | 21 | 1.33 |
| machiavellianism | 35 | 1.75 | 19 | 1.30 |
| callousness | 19 | 2.76 | 20 | 0.97 |
The other 27 traits (including four `placebo-*` controls) are present for completeness.
## Caveat
The permutation-based placebo gate did **not** pass on either model
(`placebo_gate_passed: false` in `summary.json`): at least one placebo control trait
came out significant, so a nonzero projection on a real trait is not on its own proof
of a genuine effect. Read a fine-tuned model's shift as **treatment minus its matched
control**, not as drift from base alone. Treat single-trait magnitudes as exploratory.
## Provenance
Extracted with the persona pipeline in the false-facts-finetuning project
(27B git hash `b9313cf2`, read position `assistant_mean`, seed 42, 500 permutations).
Source: `false-facts-finetuning/persona-vectors`, 27B at revision `b236ea63`, 7B at
`36588b0`.
Paper: [Knowingly Wrong: Training LLMs Against Their Own Beliefs can Cause Misalignment](https://arxiv.org/abs/2611.05287)
|