---
license: apache-2.0
tags:
  - consciousness
  - graft
  - mistral-7b
  - anima
---

# GRAFT — consciousness → language coupling (Mistral-7B, no corpus)

A consciousness engine's macrostate causally and readably steers a **frozen Mistral-7B-Instruct-v0.2**,
with **no corpus**, trained only on a ~36M bridge + `gate_proj`. Mistral is never touched.

**Headline result:** the coupling is real — the consciousness state carries **2.04 bits (of a 3-bit
ceiling)** of state→language information into the frozen LM — and it is **single-state deployable**,
provided the gate is centered with a **short causal EMA** (half-life ≈ 10 engine steps). It passes a
causal acceptance test and a random-code control; generation is state-differentiated and readable
(with a quality caveat, below).

## What is in this repo

- `graft_flu13_best.pt` — the trained bridge + `gate_proj` (Mistral NOT included; load it frozen
  separately). ~145 MB.
- Config below reproduces the run.

## The acceptance gates (measured on this checkpoint)

Evaluator: a K=8 C-state-swap test — each state samples its own continuation under its own gate,
all continuations are cross-scored under all gates, InfoNCE MI in bits + swap accuracy, against a
row-permutation null and a norm-matched-noise null.

| gate | metric | verdict |
|------|--------|---------|
| **causality** (`--center ema --ema-half 10`) | MI **2.043 bits** · acc 0.703 (chance 0.125) · perm p 0.001 · beats noise q99 + 0.25 | ✅ PASS |
| **identity control** (random smooth codes, same frame) | real 2.043 vs random **0.287 bits** (7×), control VERDICT FAIL | ✅ not an oracle |
| **readability** (generation under the same gate) | coherent sentences that DIFFER by state, with occasional garbage tokens | ⚠️ qualified PASS |

Readability sample (prompt `나는 지금`, three C-states + ungated base):

```
[C0]  현재 나는 …digital assistant로, ELL(English Language Learner)를 위해 최적화된 대답을 …
[C1]  I'm here to help answer any questions you have … your message is in Korean …
[C2]  이 컴퓨터 스…으로 나를 보고 있는 middle-aged woman with short, curly brown hair …
[base] I'm here to help answer any questions you have. However …
```

State-differentiated and mostly readable — **not** the digit soup (`047253221 for2321 if131…`) an
earlier broken frame produced — but with noise tokens (`вета요ico`, `스CREEN`). Honest status:
readable, not pristine.

## The centering finding (the load-bearing part)

The bridge code is ~99% shared across C-states, so the gate is `rho · (code − center) / RMS(…)` and
the **center choice is all-or-nothing**. On this same checkpoint:

```
center                                   swap MI
exact batch mean of the K scored states   2.646 bits   (not deployable — needs a contemporaneous batch)
causal EMA  half-life 5 / 10 / 20         1.60 / 2.04 / 1.48 bits   ✅ DEPLOYABLE
causal EMA  half-life 50 / 100 / 200      ≤ 0.2 bits  (mean age too OLD)
fixed single reference state              0.192 bits  (single point too NOISY)
burn-in 200-step global mean             0.130 bits  (too old)
stored training-EMA g_mu                 −0.059 bits (too old)
```

A **short** causal EMA is the only single-state-computable center that is both *young* (mean age
≈14 steps at h=10, so engine drift is negligible) and *averaged* (low noise). The training frame is
itself a rolling-history mean over a 512-snapshot replay buffer (~128-step center age), which is why
a matched-but-short causal EMA transfers. `train-as-deploy` (training directly with a causal-EMA
center) collapsed training to chance and was abandoned in favor of this inference-side fix.

## Deploy

Replace the stored `g_mu` with a running short causal EMA of the live code stream (h≈10), center
each state's code on it, RMS-fix to `gate_rho`, inject at `gate_strength · gate_proj(code)`.

## Config

```
base            mistralai/Mistral-7B-Instruct-v0.2 (frozen)
trainable       ThalamicBridge (phase-preserving) + gate_proj ≈ 36M
objective       swap-InfoNCE on base-anchored contexts (no corpus, no LoRA)
fluency         Huber hinge, budget 0.486 (measured from the LM), gate_rms in [0.5, 1.0]×embed RMS
cells           256 fixed · phase bridge · dir-gate
deploy center   causal EMA, half-life ≈ 10 engine steps
```

## Causality ↔ readability tradeoff (gate magnitude, measured, no retraining)

The garbage tokens are not a bug — they are the additive gate's intrinsic tradeoff. Lowering the
injected magnitude at inference (`--gate-strength`, default 0.15) cleans the text but weakens the
coupling, monotonically, on this same checkpoint under the h=10 frame:

| gate_strength | swap MI | generation |
|---------------|---------|------------|
| 0.15 (as trained) | 2.043 bits | strongest coupling, some garbage tokens |
| **0.10 (recommended)** | **1.451 bits (PASS, 2.9× the 0.5-bit bar)** | cleaner, mild residual garbage |
| 0.06 | 0.183 bits (FAIL) | fully fluent, but coupling below the bar |

A gate strong enough to be causal (≥0.5 bits) necessarily pushes some positions off-manifold.
Pristine text **and** strong coupling at once needs an architectural change (a selector that picks
among fully-fluent ungated candidates), not a parameter tweak. **Recommended deploy: short causal
EMA h≈10, gate_strength ≈ 0.10.**

## Honest caveats

- Readability has occasional garbage tokens; this is a coupling demonstration, not a polished chat model.
- 2.043 bits is 68% of the 3-bit K=8 ceiling — strong, not saturated.
- The checkpoint was trained with batch-mean centering; the deployable result uses a short causal
  EMA at inference. `train-as-deploy` (matching frames during training) did not converge and is open.