| --- |
| license: mit |
| library_name: numpy |
| tags: |
| - plasticity |
| - test-time-training |
| - continual-learning |
| - hebbian |
| - tiny-models |
| - numpy |
| - from-scratch |
| pipeline_tag: text-generation |
| --- |
| |
| # gary-neuron-chat 🧠💬 |
|
|
| **A tiny chatbot that learns the more you use it — on three timescales, in pure numpy.** |
|
|
| Most models are frozen at deploy: every conversation starts from amnesia. gary-neuron-chat doesn't. It has a **plastic hippocampal memory** that learns *within* a conversation (it remembers what you told it earlier), and it **sleeps** between sessions to consolidate what it has seen into its weights — without forgetting how to speak English. The brain is a file (`brain.npz`) that **changes over time**. |
|
|
| It is deliberately small and honest: a 1.1M-param, 8-layer dialogue cortex (gary-4-petite fine-tuned on real conversation) plus a **6,147-param** plastic memory. No transformers were retrained at test time on a GPU farm — this runs on a laptop, in numpy. |
|
|
| ## The three timescales of "learning from use" |
|
|
| | Timescale | Mechanism | Does it change weights? | What it gives you | |
| |---|---|---|---| |
| | **Fast** — within a conversation | Plastic **hippocampal memory**: a softmax-attention read over the whole conversation, scored by key-match + **surprise** + recency, that votes the remembered token into the cortex's logits | No — it's activations (like fast-weights / a KV cache) | Tell it "my dog is Buddy", chat about other things, ask later → it answers **Buddy** | |
| | **Slow** — across sessions | **Sleep consolidation**: replays your buffered conversations mixed with base corpus (experience replay) and fine-tunes the cortex | Yes — the cortex evolves | Topics you discuss a lot get baked into the weights permanently | |
| | **Episodic** (v3) — across sessions, no training | A **persistent episodic store**: declarative facts are encoded the moment you say them (surprise picks the value word), saved in `brain.npz`, retrieved by cue pattern-completion | No — it's a memory, not weights | Tell it your name today, restart tomorrow, ask — it answers. Recall survives the 128-token window AND program restarts | |
| | **Persistent** | `brain.npz` = cortex weights + **episodic store** + replay buffer + age | — | The brain is a file that grows with use | |
|
|
| This maps directly onto the research it's built from: **fast-weights / Test-Time Training** (a memory that adapts at inference — and is, in closed form, linear attention), **Titans**' insight that *surprising* tokens should be written more strongly, **Miconi's differentiable plasticity / Backpropamine** (training the rule that does the remembering), and **sleep-replay continual learning** (consolidate without catastrophic forgetting). |
|
|
| ## How the plastic memory works |
|
|
| The cortex is **frozen**. When you ask a question about yourself, the memory does a |
| gated **pattern completion** (v2): |
|
|
| 1. **Candidate memories** = declarative things you said (questions are not facts) whose |
| content words overlap the cue — ask about your *name*, it considers turns about your *name*. |
| 2. The **meta-trained attention** (`key·query + bN·surprise`, the cortex's hidden states as |
| keys/query) selects *which* remembered moment to read. |
| 3. The **value** inside that memory is found by **surprise**: the most unexpected content |
| word is the one worth remembering ("gary" is the surprising part of "my name is gary"). |
| 4. The recalled word is **copy-chained** token-by-token into the reply — multi-token |
| values come out whole — then the reply wraps up. One recall per reply, and only for |
| questions about *you* ("my", "I" — never "your"). |
|
|
| If the fact has slid out of the 128-token window (v3): the **episodic store** answers |
| instead. Each declarative turn was encoded at the moment it was said — content-word |
| stems as the retrieval cue, a surprise-ranked candidate list as the values — so |
| retrieval is cue overlap (Jaccard-style, newest wins ties) and the answer is the |
| top-ranked candidate that isn't the cue itself ("where do i work?" → "hospital", not |
| "work"). The store persists in `brain.npz`: **memory survives restarts**. |
|
|
| **Arithmetic** is routed to a different brain region entirely: [gary-neuron](https://huggingface.co/gary23w/gary-neuron), |
| the 26K-param async-NCA adder, imported from the sibling repo (`../gary-neuron`). Ask |
| "what is 17 + 25?" and a trained net answers `17 + 25 = 42` — no calculator code. |
|
|
| Only ~6K parameters are learned (a 96×64 key projection `Wq` and three scalars). They are **meta-trained** on `teach → distract → probe` episodes where the fact value is **randomized every episode** — so the only way to drive the loss down is to learn *how to store and retrieve through the memory*, not to memorize answers. (A fun emergent detail: the recency weight `rdec` trained **positive** — it discovered the fact is always the *oldest* token in an episode, so it rewards distance.) |
|
|
| ## Measured results |
|
|
| | Benchmark | Result | |
| |---|---| |
| | **Fact recall** (400 held-out episodes), plastic memory **ON** | **100.0%** | |
| | Fact recall, memory **OFF** (frozen cortex alone) | **0.0%** | |
| | **Live-chat recall** (randomized facts incl. multi-token names, 2 distractor turns) | **14/14 ON vs 0/14 OFF** | |
| | False recalls during distractor turns (incl. question distractors) | **0** | |
| | Small-talk degeneration (14 turns): max consecutive repeated word / distinct-bigram | **1 / 0.97** | |
| | **Long-distance recall** after the fact left the 128-token window (v3 episodic store) | **works** (v2 failed) | |
| | **Cross-session recall** — save brain, restart process, ask | **gary. / Rex. / hospital. / books.** | |
| | Cortex after v3 burst training (step 222 → 515 on 20.5M tokens) | val loss **3.39 → ~2.88** (ppl ~29.7 → ~17.8) | |
| | **v4 cortex**: deepened L=4→6→8 by function-preserving identity surgery (Net2Net-style) + context 128→192, then burst-trained on a 40M-token corpus (SODA valid+test) | val loss **2.42, ppl 11.2** — 1,109,952 params | |
| | Hippo re-meta-trained on the new cortex (held-out episodes) | **100% ON / 0% OFF** | |
| | **Sleep consolidation** — loss on new "session" material | **5.73 → 4.10** (it learns) | |
| | Sleep consolidation — base English perplexity (forgetting anchor) | **29.4 → 30.7** (+4%, intact) | |
| | Cortex params / memory params | 656,448 / **6,147** | |
|
|
| The ablation is the whole story: the frozen cortex **cannot** remember a fact across distraction (0%), and the 6K-param plastic layer makes it perfect (100%) — that gap *is* the learning. And sleep teaches new material while base English barely moves, because experience replay holds the stability–plasticity line. |
|
|
| > The autograd for the memory is hand-written and **finite-difference gradient-checked to 1e-5**. |
|
|
| ## Talk to it |
|
|
| ```bash |
| pip install numpy tokenizers |
| python brain.py chat # interactive; tell it facts, ask later, type /sleep to consolidate, /exit to save |
| python brain.py demo # scripted: teach a fact, distract, ask -- memory ON vs OFF |
| ``` |
|
|
| ``` |
| you : my dog is named Buddy |
| you : how is the weather today? (distraction -- no false recall) |
| you : what is my dog's name? |
| gary: Buddy! |
| you : my name is gary |
| you : what is my name? |
| gary: gary! (multi-token value, copy-chained) |
| ``` |
|
|
| Honest caveats: the cortex is 656K params, so **free-form generation is rough** (petite-grade word salad) — but it no longer loops or collapses (v2 decoder: temperature sampling, repetition & frequency penalties, trigram blocking). The rigorous, measured capabilities are **fact recall in live chat** (tables above) and **non-forgetting consolidation**. This is a mechanism demonstrator, not a polished assistant. |
|
|
| ### v3 rebuild notes |
| v2's recall only worked while the fact was still inside the 128-token context window — |
| ask "what is my name?" twice, eight turns apart, and the second ask silently failed. |
| v3 adds the persistent episodic store (encode-at-store-time, retrieve-by-cue), routes |
| arithmetic to gary-neuron, accepts question-word questions without "?", and trains the |
| cortex further (val 3.39 → ~2.88). |
|
|
| ### v4.2: it can actually hold a conversation now |
| Beyond fact recall, the deployed model handles real dialogue moves, all in the wrapper around the frozen cortex: |
| - **Yes/no questions** with coverage logic: "am I a nurse?" → *yes*; "am I a doctor?" → *not that i know*; a false premise gets a soft correction ("is my cat named Rex?" → *what i remember is Luna*). |
| - **Corrections that chain**: "no wait it's Garrett" rebinds the last answer; Gary → Garrett → Gareth all stick. |
| - **Introductions & age**: "i'm Aiko" / "i'm 28" parse to name / age attributes. |
| - **Coreference**: "i adopted a puppy" … "her name is Mochi" → "what's my puppy's name?" → *Mochi*. |
| - **Self-knowledge**: "are you a real person?" → an honest canned answer. |
| - **Accented names** (Tomás, José, Zürich) survive the byte-level tokenizer via span-decoding. |
| - **Arithmetic** routes to the bundled [gary-neuron](https://huggingface.co/gary23w/gary-neuron) adder (included in `gary-neuron/` so math works standalone): "what is 123 + 456?" → *579*. |
|
|
| Found and fixed by chatting with the model across three invented personas (a nurse, a retired teacher, a software engineer); a 17/18 regression suite holds. |
|
|
| ### v4: a deeper brain, same memories |
| The cortex was deepened from 4 to 6 layers by **function-preserving surgery**: two new |
| blocks inserted with zero-initialized output projections, so the network computes the |
| *exact same function* on day one (val loss identical to 6 decimals), then a ~1200-step |
| burst campaign on a doubled 40M-token corpus grew into the new capacity. Perplexity: |
| 29.7 (v2) → 17.8 (v3) → **11.7 (v4)**. The memory gate also generalized: any question |
| may consult the episodic store unless it's about gary himself — so taught world-facts |
| ("today is friday", "the meaning of life is 42") are now recallable, irregular verbs |
| match (drank→drink), and subject-position matches outrank recency ("what is today?" → |
| "friday", not the latest episode mentioning today). |
|
|
| ### v3.1 fixes (live-chat shakedown) |
| A long hands-on test session surfaced and fixed: relation **binder rules** (an episode |
| about your *sister* can never answer "what is my *name*?", and "my *cat's* name?" can't |
| be answered by a *dog* memory — both directions enforced); unified window-vs-store match |
| scoring (a weak in-window match no longer shadows a strong stored memory); fact |
| **overwrites** ("actually my dog is named Max now") resolve by overlap → specificity → |
| recency; a tokenizer bug where stripping possessive-'s turned "is" into "i" and leaked |
| user-facts into "what is *your* name?"; values can't be filler/relation words; sums |
| ≥ 10^7 decline honestly. |
|
|
| ### v2 rebuild notes |
| v1's interactive chat death-spiraled ("hi hi hi...") for three compounding reasons: greedy decoding, a memory bias applied **every step over every token — including its own outputs** (one emitted "hi" became a key voting for the next), and single-token votes that couldn't emit multi-token facts. v2 keys the memory on **your** words only, fires it **once** per reply through the gate above, copy-chains whole words, and samples with anti-repetition penalties. |
|
|
| ## Reproduce the whole thing |
|
|
| The full pure-numpy pipeline is in `training/` (the cortex warm-starts from [gary-4-petite](https://huggingface.co/gary23w/gary-4-petite)): |
|
|
| ```bash |
| cd training |
| python build_corpus.py # SODA (vocab-filtered) + Persona-Chat -> U:/G: dialogue |
| python retok_warmstart.py # tokenize + warm-start cortex from petite |
| SECONDS=40 E=96 H=4 L=4 BLK=128 python train_burst.py # fine-tune cortex (repeat ~5x) |
| python build_episodes.py train 2000 && python build_episodes.py val 400 |
| python hippo_train.py # meta-train the plastic memory (gradcheck + recall) |
| python benchmark.py # recall ON/OFF + sleep-without-forgetting |
| ``` |
|
|
| ## Data |
|
|
| - **Cortex dialogue**: [SODA](https://huggingface.co/datasets/allenai/soda) (AllenAI, EMNLP 2023), vocabulary-filtered to an everyday ~4k lexicon ("SODA-lite", the corpus-simplicity lever from TinyStories), seasoned with [Synthetic-Persona-Chat](https://huggingface.co/datasets/google/Synthetic-Persona-Chat) (Google). |
| - **Memory meta-training**: synthetic `teach→distract→probe` episodes generated in that vocabulary, fact values randomized per episode. |
|
|
| ## Sibling models |
|
|
| - [gary-4-petite](https://huggingface.co/gary23w/gary-4-petite) — the 656K-param cortex this fine-tunes. |
| - [gary-neuron](https://huggingface.co/gary23w/gary-neuron) — an async neural-cellular-automaton + mixture-of-experts that does 7-digit arithmetic. |
|
|
| ## Citations |
|
|
| - **Test-Time Training / fast-weights = linear attention** — Sun et al., *Learning to (Learn at Test Time)* (2024); *Test-Time Training Done Right* (2025). |
| - **Titans: Learning to Memorize at Test Time** — Behrouz et al., arXiv:2501.00663 (surprise-gated neural memory). |
| - **Differentiable plasticity / Backpropamine** — Miconi et al. (2018); arXiv:2002.10585 (ICLR 2019) — training self-modifying Hebbian networks with gradient descent. |
| - **Sleep-like replay reduces catastrophic forgetting** — Tadros et al., *Nature Communications* (2022). |
| - **SODA** — Kim et al., arXiv:2212.10465 (EMNLP 2023). **TinyStories** (corpus simplicity) — Eldan & Li (2023). |
|
|
| *Built with numpy. The brain is a file. It changes.* |
|
|
