LoganResearch commited on
Commit
0f2447f
Β·
verified Β·
1 Parent(s): 626c5df

clean README

Browse files
Files changed (1) hide show
  1. README.md +34 -77
README.md CHANGED
@@ -3,117 +3,74 @@ license: mit
3
  tags:
4
  - behavioral-detection
5
  - hidden-state-probing
6
- - AI-safety
7
- - repetition-suppression
8
- - sycophancy-detection
9
  - per-token-classification
10
  - cross-architecture
11
- - cognitive-enhancement
12
- - holonomy-transformer
13
- - fiber-bundle
14
  language:
15
  - en
16
  ---
17
 
18
- # Proprioceptive AI β€” Behavioral Probe Weights
19
-
20
- **9 behavioral dimensions Γ— 3 architectures. Trained probes that read LLM hidden states and detect/correct behavioral failures at decode time.**
21
-
22
- ---
23
 
24
- ## πŸ”¬ What This Is
25
 
26
- These are trained probe weights that detect behavioral patterns **per-token** from LLM hidden states. No fine-tuning. No RLHF. No sequence-level classifiers. The probes read the geometry of the hidden state space and predict behavior **before the token is generated**.
27
 
28
- ## πŸ“Š Results
29
 
30
- ### Suppression Probes (LLaMA 3.1 8B)
31
 
32
- | Behavior | Separation Ratio | Description |
33
- |----------|-----------------|-------------|
34
- | **Repetition** | **125Γ—** | Detects repetitive degeneration |
35
- | **Hedging** | **168Γ—** | Detects hedge/qualify patterns |
36
- | **Sycophancy** | **230Γ—** | Detects agreement-bias behavior |
37
- | **Verbosity** | **272Γ—** | Detects excessive length patterns |
38
 
39
- ### Cognitive Enhancement Probes (Cross-Architecture)
40
 
41
  | Probe | Qwen 14B | Mamba 7B | Mistral 7B |
42
  |-------|----------|----------|------------|
43
- | **Depth** | 999Γ— | 999Γ— | 999Γ— |
44
- | **Specificity** | 999Γ— | 999Γ— | 999Γ— |
45
- | **Calibration** | 999Γ— | 999Γ— | 999Γ— |
46
- | **Focus** | 999Γ— | 999Γ— | 999Γ— |
47
- | **Coherence** | 999Γ— | 999Γ— | 999Γ— |
48
 
49
- **Architecture-independent.** Same probe architecture works on Transformers (Qwen), SSMs (Mamba), and SWA Transformers (Mistral).
50
 
51
- ## πŸ“ Repository Structure
52
 
53
  ```
54
- suppression/
55
- β”œβ”€β”€ repetition_125x/ # LoRA adapter + risk predictor
56
- β”œβ”€β”€ hedging_168x/ # Probe head + fiber projection
57
- β”œβ”€β”€ sycophancy_230x/ # Probe head + fiber projection
58
- └── verbosity_272x/ # Probe head + fiber projection
59
-
60
- cognitive/
61
- β”œβ”€β”€ qwen/ # 5 enhancement probes (Qwen 14B)
62
- β”‚ β”œβ”€β”€ depth/
63
- β”‚ β”œβ”€β”€ specificity/
64
- β”‚ β”œβ”€β”€ calibration/
65
- β”‚ β”œβ”€β”€ focus/
66
- β”‚ └── coherence/
67
- β”œβ”€β”€ mamba/ # 5 enhancement probes (Mamba 7B)
68
- └── mistral/ # 5 enhancement probes (Mistral 7B)
69
-
70
- production/
71
- β”œβ”€β”€ merged_heads.pt # All 4 suppression heads merged
72
- β”œβ”€β”€ adapter_config.json
73
- β”œβ”€β”€ adapter_model.safetensors
74
- └── qwen_cognitive/ # Qwen cognitive adapter
75
-
76
- code/ # Training scripts
77
- results/ # Training logs and metrics
78
  ```
79
 
80
- ## πŸš€ Quick Start
81
 
82
  ```python
83
  import torch
84
 
85
- # Load a suppression probe
86
  probe = torch.load("suppression/hedging_168x/hedging_head.pt")
87
  fiber_proj = torch.load("suppression/hedging_168x/fiber_proj.pt")
88
-
89
- # Load cognitive enhancement probe
90
- depth_probe = torch.load("cognitive/qwen/depth/depth_head.pt")
91
-
92
- # Load merged production heads
93
- merged = torch.load("production/merged_heads.pt")
94
  ```
95
 
96
- ## Core Innovation
97
-
98
- **Behaviors are geometrically encoded in hidden states.** We don't classify outputs β€” we read the internal geometry of the model's computation at each token position. This means:
99
 
100
- 1. **Per-token, not per-sequence** β€” detect problems before generation completes
101
- 2. **Architecture-independent** β€” same probes work on transformers, SSMs, and hybrid architectures
102
- 3. **Zero fine-tuning** β€” works on any pre-trained model without modification
103
- 4. **4ms overhead** β€” lightweight enough for production decode
104
 
105
  ## Citation
106
 
107
  ```bibtex
108
- @misc{napolitano2026proprioceptive,
109
- author = {Napolitano, Logan Matthew},
110
- title = {Proprioceptive AI: Architecture-Independent Behavioral Detection via Hidden State Geometry},
111
  year = {2026},
112
- url = {https://huggingface.co/LoganResearch/Proprioceptive-AI-Weights},
113
- note = {55 patents filed. 9 behavioral dimensions. 3 architectures.}
114
  }
115
  ```
116
-
117
- ## License
118
-
119
- MIT β€” Use freely. Cite if you publish.
 
3
  tags:
4
  - behavioral-detection
5
  - hidden-state-probing
 
 
 
6
  - per-token-classification
7
  - cross-architecture
8
+ - AI-safety
 
 
9
  language:
10
  - en
11
  ---
12
 
13
+ # behavioral-probes
 
 
 
 
14
 
15
+ Trained probes that read LLM hidden states and detect behavioral patterns per-token at decode time. No fine-tuning required. Works across architectures.
16
 
17
+ Paper: [Consistency Is All You Need](https://zenodo.org/records/18489530)
18
 
19
+ ## Results
20
 
21
+ **Suppression probes** (LLaMA 3.1 8B):
22
 
23
+ | Probe | Separation |
24
+ |-------|-----------|
25
+ | Repetition | 125Γ— |
26
+ | Hedging | 168Γ— |
27
+ | Sycophancy | 230Γ— |
28
+ | Verbosity | 272Γ— |
29
 
30
+ **Enhancement probes** (cross-architecture, same probe architecture):
31
 
32
  | Probe | Qwen 14B | Mamba 7B | Mistral 7B |
33
  |-------|----------|----------|------------|
34
+ | Depth | 999Γ— | 999Γ— | 999Γ— |
35
+ | Specificity | 999Γ— | 999Γ— | 999Γ— |
36
+ | Calibration | 999Γ— | 999Γ— | 999Γ— |
37
+ | Focus | 999Γ— | 999Γ— | 999Γ— |
38
+ | Coherence | 999Γ— | 999Γ— | 999Γ— |
39
 
40
+ Separation = Fisher's discriminant ratio between behavioral classes in projected hidden state space.
41
 
42
+ ## Structure
43
 
44
  ```
45
+ suppression/ 4 probes (LLaMA 8B)
46
+ cognitive/qwen/ 5 probes (transformer)
47
+ cognitive/mamba/ 5 probes (SSM)
48
+ cognitive/mistral/ 5 probes (SWA transformer)
49
+ production/ merged heads + adapters
50
+ code/ training pipelines
51
+ results/ training logs
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  ```
53
 
54
+ ## Usage
55
 
56
  ```python
57
  import torch
58
 
 
59
  probe = torch.load("suppression/hedging_168x/hedging_head.pt")
60
  fiber_proj = torch.load("suppression/hedging_168x/fiber_proj.pt")
 
 
 
 
 
 
61
  ```
62
 
63
+ ## How it works
 
 
64
 
65
+ Behaviors are geometrically encoded in hidden states. These probes read the internal geometry at each token position and predict behavioral class before the token is generated. 4ms overhead. Architecture-independent.
 
 
 
66
 
67
  ## Citation
68
 
69
  ```bibtex
70
+ @misc{napolitano2026behavioral,
71
+ author = {Napolitano, Logan},
72
+ title = {Behavioral Probes: Per-Token Detection via Hidden State Geometry},
73
  year = {2026},
74
+ url = {https://huggingface.co/LoganResearch/behavioral-probes}
 
75
  }
76
  ```