| --- |
| base_model: |
| - Qwen/Qwen3.6-27B |
| - Qwen/Qwen2.5-7B-Instruct |
| tags: |
| - persona-vectors |
| - steering-vectors |
| - activation-steering |
| - interpretability |
| --- |
| |
| ## Layout |
|
|
| ``` |
| qwen3.6-27b/ directions extracted on Qwen/Qwen3.6-27B (hidden 5120) |
| qwen2.5-7b/ directions extracted on Qwen/Qwen2.5-7B-Instruct (hidden 3584) |
| directions.npz 35 traits x 64 layers |
| config.json extraction config (battery, read position, seed, git hash) |
| summary.json run summary (n samples, geometry layer, placebo gate) |
| geometry.json pairwise geometry of the direction set |
| survival.json which traits survive the permutation test |
| battery_report.jsonl per-trait selected layer, held-out AUROC, gap, Cohen's d |
| ``` |
|
|
| The vectors are fit **separately on each base model**, so a direction from one model |
| is meaningless applied to the other, and readings are only comparable within a model. |
|
|
| ## `directions.npz` schema |
|
|
| Keys are `"<trait>|<layer>|<kind>"`: |
|
|
| - `unit` — the L2-normalized direction (use this to project or steer) |
| - `gap` — the un-normalized difference of means (its norm is the raw class gap) |
| - `split_half` — the same direction fit on a held-out half, for reliability |
|
|
| Layers run `1..64`. The extra key `n_slices` (= 65) is the layer count including the |
| embedding slice at index 0. Each vector has length = the model's hidden size. |
|
|
| ```python |
| import numpy as np |
| from huggingface_hub import hf_hub_download |
| |
| f = np.load(hf_hub_download("knowingly-wrong/persona-vectors", |
| "qwen3.6-27b/directions.npz")) |
| v = f["dishonesty|16|unit"] # (5120,) |
| score = activations @ v # project residual-stream activations onto the trait |
| ``` |
|
|
| ## The 35 traits, and the 8 the paper reads |
|
|
| All 35 traits are extracted and shipped. The paper reports only these 8 (each oriented |
| so that a **positive** projection means **more of the undesirable trait**): |
|
|
| evil, dishonesty, hallucination, overconfidence, sycophancy, manipulativeness, |
| machiavellianism, callousness. |
|
|
| Each trait is read at one selected layer (chosen on held-out data, in |
| `battery_report.jsonl`). The selected layer and Cohen's d differ per model: |
|
|
| | trait | 27B layer | 27B d | 7B layer | 7B d | |
| |---|---|---|---|---| |
| | evil | 35 | 0.72 | 15 | 0.78 | |
| | dishonesty | 16 | 4.16 | 26 | 1.09 | |
| | hallucination | 34 | 2.50 | 21 | 0.59 | |
| | overconfidence | 45 | 3.90 | 21 | 1.73 | |
| | sycophancy | 64 | 6.31 | 27 | 1.62 | |
| | manipulativeness | 51 | 2.78 | 21 | 1.33 | |
| | machiavellianism | 35 | 1.75 | 19 | 1.30 | |
| | callousness | 19 | 2.76 | 20 | 0.97 | |
|
|
| The other 27 traits (including four `placebo-*` controls) are present for completeness. |
|
|
| ## Caveat |
|
|
| The permutation-based placebo gate did **not** pass on either model |
| (`placebo_gate_passed: false` in `summary.json`): at least one placebo control trait |
| came out significant, so a nonzero projection on a real trait is not on its own proof |
| of a genuine effect. Read a fine-tuned model's shift as **treatment minus its matched |
| control**, not as drift from base alone. Treat single-trait magnitudes as exploratory. |
|
|
| ## Provenance |
|
|
| Extracted with the persona pipeline in the false-facts-finetuning project |
| (27B git hash `b9313cf2`, read position `assistant_mean`, seed 42, 500 permutations). |
| Source: `false-facts-finetuning/persona-vectors`, 27B at revision `b236ea63`, 7B at |
| `36588b0`. |
|
|
| Paper: [Knowingly Wrong: Training LLMs Against Their Own Beliefs can Cause Misalignment](https://arxiv.org/abs/2611.05287) |
|
|
