celiumsAI commited on
Commit
89180a5
·
verified ·
1 Parent(s): b668c2b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +86 -0
README.md ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: gpl-3.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - proprioception
7
+ - cognitive-control
8
+ - conditioning
9
+ - interpretability
10
+ - gemma
11
+ - adapter
12
+ - cross-attention
13
+ - rezero
14
+ pipeline_tag: text-generation
15
+ base_model: google/gemma-4-E2B-it
16
+ ---
17
+
18
+ # tinyMARS — Proprioceptive Channels
19
+
20
+ **A second, perpendicular input to a language model: six cognitive self-state channels that the model
21
+ learns to obey — even against the text prompt.**
22
+
23
+ Research from [**Celiums Research Labs**](https://celiums.ai) (a division of Celiums Solutions, LLC).
24
+
25
+ > **Paper:** *Proprioceptive Channels: Cognitive Self-State as a Perpendicular Control Axis in Language
26
+ > Models* · [PDF](https://celiums.ai/papers/tinymars-proprioceptive-channels.pdf) ·
27
+ > [DOI 10.5281/zenodo.20531347](https://doi.org/10.5281/zenodo.20531347) ·
28
+ > [Code (GitHub)](https://github.com/terrizoaguimor/tinymars)
29
+
30
+ ## TL;DR
31
+
32
+ A decoder-only LM is normally a single-channel structure: text in, text out. We add a **perpendicular**
33
+ input — six cognitive self-state channels (**memory, affect, time, ethics, identity, continuity**) injected
34
+ at every layer via per-channel **gated cross-attention with ReZero** — and call it *proprioception*, by
35
+ analogy to the body's sense of its own configuration.
36
+
37
+ **The load-bearing result (measured, judge-free):** under direct conflict — where the channel asserts one
38
+ state and the text prompt asserts the opposite — generation follows the **channel 264/265 times (98–100%)**.
39
+ A single-channel (text-only) model cannot exhibit this. The channels are also **causal** (six coexist in one
40
+ model with no interference, 6/6) and **bit-exact to the base at initialization** (ReZero α=0 ⇒ zero delta).
41
+
42
+ ## Two experiments
43
+
44
+ 1. **Adapter on a frozen base.** A ~186M-parameter channel adapter on a frozen **Gemma 4 E2B-it**. Frozen
45
+ base ⇒ this is the *channels-over-Gemma* result; identity/attribution stays with Google's Gemma + the
46
+ Celiums channel adapter.
47
+ 2. **Native from scratch.** A 110M-parameter decoder trained from random init with channels present from
48
+ layer 1; the perpendicular force reproduces from scratch (conflict-win 0.888 on held-out, chance 0.25),
49
+ with a clean attributed relief valve. Honest scope: a toy-scale *property*, not a product-scale claim.
50
+
51
+ ## How it works
52
+
53
+ ```
54
+ hidden ──► gated cross-attention (per channel) ──► Σ αᵢ · ctxᵢ ──► + residual
55
+ channels ──► [memory 1024 · affect 2 · time 16 · ethics 24 · identity 1024 · continuity 1024]
56
+ α (ReZero gates) init 0 ⇒ delta = 0 ⇒ bit-exact passthrough until trained
57
+ ```
58
+
59
+ The adapter trains while the base stays frozen; only the cross-attention projections and the ReZero gates
60
+ move. `alpha_l2` (the L2 norm of the gates) growing from 0 is the signal that the model is *using* the
61
+ channels.
62
+
63
+ ## Use / reproduce
64
+
65
+ The adapter, training, and evaluation code (with the channel-causal eval suite — counterfactual,
66
+ judge-free) are in the [GitHub repository](https://github.com/terrizoaguimor/tinymars). The native
67
+ checkpoints and the corpus generators are described there. This page is the research companion; see the
68
+ paper for the full method and the honest negatives.
69
+
70
+ ## Citation
71
+
72
+ ```bibtex
73
+ @misc{gutierrez2026proprioceptive,
74
+ title = {Proprioceptive Channels: Cognitive Self-State as a Perpendicular Control Axis in Language Models},
75
+ author = {Gutierrez, Mario},
76
+ year = {2026},
77
+ publisher = {Celiums Research Labs},
78
+ doi = {10.5281/zenodo.20531347},
79
+ url = {https://github.com/terrizoaguimor/tinymars}
80
+ }
81
+ ```
82
+
83
+ ## License
84
+
85
+ Code: **GPL-3.0**. Paper & docs: **CC-BY-SA-4.0**. The frozen base model (Gemma 4) is subject to Google's
86
+ Gemma terms; this work distributes the **channel adapter and method**, not Gemma's weights.