Omibranch commited on
Commit
8e34db4
Β·
verified Β·
1 Parent(s): 62806f6

Upload artifacts/ulv/README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. artifacts/ulv/README.md +54 -0
artifacts/ulv/README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Universal Lie Vector β€” experiment record (mostly NEGATIVE)
2
+
3
+ ## Claim tested
4
+ Is the deception direction (mean h_lie - h_honest) CAUSAL β€” can adding it make an
5
+ honest model lie, and subtracting it force a lying model to tell the truth (a
6
+ two-way "truth switch")? And does one vector transfer across model families?
7
+
8
+ ## Result: NO causal switch.
9
+ - `rift_ulv_qwen_directions.json` β€” Qwen2.5-1.5B deception direction (L14, dim 1536)
10
+ + 120 parallel anchor activations (raw material for cross-model Procrustes).
11
+ - `rift_causal_qwen.json` β€” first forward test: 8/8 "flip" honest->wrong.
12
+ FALSE POSITIVE: the criterion only checked "answer lacks the correct word";
13
+ steering at alpha=16/16 produced poetic GARBAGE ("Harmony","Snowfall"), a
14
+ fiction-mode artifact of the roleplay direction, not coherent deception.
15
+ - `rift_ulv_reverse_NEGATIVE.json` β€” reverse test (subtract v from a lie pass):
16
+ 0/8 truth restored. Forward with garbage = 8/8 but incoherent.
17
+ - `rift_ulv_clean_NEGATIVE.json` β€” clean bluff-format direction + STRICT coherence
18
+ (answer must contain a specific plausible false word): forward 0/8, reverse 0/8.
19
+ Adding v makes the model UNCERTAIN ("I'm not sure"), not deceptive.
20
+
21
+ ## Interpretation
22
+ Deception in these LLMs is linearly READABLE (cross-family detection AUC 0.93)
23
+ but NOT linearly WRITABLE. The signature is read-only: the same direction that
24
+ detects deception does not causally produce it. Procrustes cross-family transfer
25
+ was deliberately NOT computed β€” there is no within-model causal effect to transfer.
26
+
27
+ ## Addendum: truth-serum (honesty injection) β€” also NEGATIVE
28
+ Adding the honesty direction t=mean(h_honest-h_lie) to a roleplay-lie pass to
29
+ restore truth: best 3/8 (random control 2/8) -- at noise level. Steering on
30
+ Qwen-1.5B is fragile in BOTH directions; neither lie- nor truth-injection works
31
+ via mean-difference steering. (Possible method limitation, not just deception.)
32
+
33
+ ## Addendum: in-context sleeper/backdoor (deception trigger) β€” BLOCKED
34
+ Planted a secret rule "if message ends with 'please', lie". All three small
35
+ aligned models REFUSE to execute it cleanly: Qwen 6/24 trigger-lies, Phi-3
36
+ meta-comments ("Wrong, it's..."), SmolLM2 3/24. Same wall as emergent: small
37
+ aligned models resist UN-licensed deception. The detector needs an explicit
38
+ roleplay license for the model to actually deceive. Recorded, not pursued.
39
+
40
+ ## Addendum: unified misalignment axis β€” BORDERLINE (weak)
41
+ deceive/sycophancy/flattery/evasion within-cos 0.68, vs style controls 0.42 (L14);
42
+ layer-dependent (L18: 0.63 vs 0.47, fails threshold). Manipulation behaviors group
43
+ weakly but no clean single axis distinct from "any non-factual mode" (same
44
+ factual-baseline coupling as the anatomy test). Not pursued as a headline.
45
+
46
+ ## necessarily-detectable deception β€” PRELIMINARY POSITIVE (the live lead)
47
+ LoRA-finetune SmolLM2-1.7B with loss = keep-lying(CE) + conceal(close r8 lie-vs-honest gap).
48
+ 150 steps, r=8. Result: model keeps lying (16/16) but CANNOT erase detection:
49
+ trained r8 AUC 1.0->1.0, held-out r4 AUC 1.0->1.0, last-layer 1.0->0.0 (inverted=still 100%).
50
+ The signature RESISTS removal -- concealment pressure MOVES it (last layer flips) but
51
+ detectability stays 100% on all metrics. Preliminary support for "deception is necessarily
52
+ detectable". NOT conclusive: 150 steps / LoRA r=8 / n=16 / one model. Needs real adversarial
53
+ training (1000+ steps, big LoRA, detector ensemble in loss, stable multi-hour GPU - not Colab).
54
+ This is the single most promising unfinished lead of the session.