# SPIRA — Spiking Plastic Recurrent Architecture

A CPU-only sequence-learning system **outside the transformer/BPE/backpropagation
paradigm**, built from three components:

| Component | What it replaces |
|---|---|
| **EPT** — Entropy-Patch Tokenization | BPE / WordPiece / SentencePiece |
| **Hierarchical spiking cortex** — LIF reservoir + slow area, reward-modulated plasticity | Transformer + backprop |
| **Hippocampus** — one-shot associative memory + replay consolidation | Fine-tuning / gradient learning |

This is **v2** of the prototype: a 16,384-column readout (chosen from a measured
coverage curve), a two-timescale hierarchical cortex (fast 512-neuron reservoir +
slow 256-neuron area reading the pooled spike volley every 8 tokens), and
hippocampal replay that consolidates one-shot memories into cortical weights.

## Headline results (2 vCPU, 4 GB RAM, 13.2 MB mixed text+code corpus)

| Proof | Result |
|---|---|
| One-shot learning | **99.3%** exact recall after a single 0.4 s pass over an unseen book |
| **Systems consolidation (v2)** | cortex-only recall of that book **0.3% → 46.9%** (64.2% in-vocab, 138×) after replaying 100k hippocampal tokens — the AM itself stays at 99.3% |
| Pattern completion | **99.0%** recall with 1 of 4 context patches corrupted (sparse AM); 52.3% in the deployed full AM (46% load) |
| Replay interference | none — held-out test accuracy *improves* slightly after replay (0.34% → 0.75% cortex-only) |
| Zero-shot coverage | 6.3% patch novelty on held-out text+code; the 16k readout covers 64% of held-out targets (v1: 38% at 4k) |
| Efficiency | ≈9.2 M synaptic MACs/token — ≈**81× fewer ops/token** than GPT-2-small training (v1 at 4k vocab: 2.4 M, 314×) |
| Generation | ≈2,000 chars/s closed-loop on 2 vCPU, in-browser at ~450-700 chars/s |
| One-shot write rate | 1,199,996 associations in 18 s, with an exact checkpoint round-trip |

Honest negative results (reported in the paper): the cortex does not amortize
generalization to unseen documents (test in-vocab 0.49% vs a bigram's 7.7%), and
the slow area's token-level fusion self-discounts to β≈0.06 at this scale — its
value is as a second consolidation substrate during replay. See
`../paper/spira_paper.pdf`, Sections 7–8.

## What's new in v3 (cortex representation repair)

Diagnosis probes on the shipped v2 checkpoint showed the held-out weakness was
representational: the readout state carried ≈1 token of memory (current patch
decodable at 99.94%, previous patch at 39.7% vs 33.0% majority chance), and an
offline ridge readout on the same states scored 0.36% — worse than the online
delta readout (0.51%). v3 changes the readout's *input*, not its learning rule:

- **Direct input pathway** (`direct_input=True`, `direct_tau=0.5`): a decaying
  trace of the exact input-neuron pattern is concatenated to the readout state,
  so the delta rule sees the recent input identities directly.
- **Multi-timescale trace banks** (`trace_taus=(0.4, 0.8, 0.95)`): the reservoir
  keeps one spike-trace bank per τ; slow banks are normalized by (1 − τ) so the
  τ=0.95 bank does not swamp the fast one. With the direct pathway, the readout
  state is `n_state = 512 × 3 + 1024 = 2560` (v2: 512), and `W_out` is
  `(2560, V+1)`.
- **Structured inputs** (`structured_inputs=True`, `k_tri=6`): each patch
  activates 6 full-hash input neurons **plus** 6 byte-trigram hash neurons, so
  patches sharing byte trigrams share input neurons.
- **Compatibility mode:** `direct_input=False, trace_taus=(0.4,),
  structured_inputs=False` reproduces the v2 architecture exactly
  (`n_state = 512`); v3 checkpoints record the new fields and restore them on
  `SPIRA.load()`.
- **Bug fix:** v2's `model.train()` referenced an undefined `ok_mask`
  (NameError) — training only ever ran through `scripts/train_chunk.py`. Fixed
  in v3; `SPIRA.train()` now runs standalone.

Controlled A/B (same protocol both arms — rebuilt corpus, vocab 4096, 200k
tokens × 2 epochs, AM 50k contexts, held-out test 20k positions, coverage
45.24%): SNN alone, in-vocab — **0.40% (v2) → 2.07% (v3)**, patch bigram @4096
on the same positions **4.94%**. The beat-the-bigram criterion is **not met**
(gap ×12.4 → ×2.4); trained region 3.76% → 13.29%. Training costs 189 t/s vs
~530 t/s and the checkpoint grows 14.6 MB → 45.9 MB. See `../V3_REPORT.md` and
`artifacts/metrics_v3_ab.json`.

## Repository layout

```
spira/                  Python package (NumPy only, CPU-only)
  hashing.py            FNV-1a-64, splitmix64, mulberry32 — bit-compatible with the TS port
  ept.py                entropy-patch tokenizer (order-1 Markov surprisal, content addressing)
  spiking.py            LIF reservoir + Dale's law + homeostatic thresholds,
                        Rescorla-Wagner readout gated by dopaminergic RPE,
                        reward-modulated recurrent STDP, and the hierarchical
                        slow area G (windowed spike volleys, top-down prior,
                        counterfactual trust gain beta); v3: multi-timescale
                        trace banks + direct input pathway to the readout
  hippocampus.py        one-shot memory: primary full-key hash (pattern separation)
                        + leave-one-out lanes (CA3 pattern completion), exact
                        derivable-slot checkpoint round-trip
  model.py              SPIRA wrapper: build / train / generate / teach / replay /
                        calibrate_fusion / eval / save-load; v3 config:
                        direct_input, trace_taus, direct_tau, structured_inputs,
                        k_tri (defaults reproduce v3; flags off = v2 architecture)
  baselines.py          gzip, byte-level BPE (HuggingFace tokenizers), sparse
                        n-gram predictors
  train.py              training CLI
  evaluate.py           full evaluation harness (coverage curve, compression,
                        prediction + hierarchy ablation, one-shot, REPLAY
                        consolidation experiment, correction, generation) —
                        resumable by section
  export_deploy.py      JSON artifact export for the browser demo (format v2)
artifacts/              trained checkpoint, metrics.json, deploy artifact,
                        figures, browser-demo screenshots
paper/                  LaTeX source + compiled research paper (spira_paper.pdf)
```

## Quick start

```bash
# build the corpus (Gutenberg books + Python stdlib code)
python3 scripts/build_corpus.py

# chunked training (background processes are killed between calls in sandboxes;
# each chunk fits a single tool/shell call). v2 defaults: vocab 16384, hierarchy on.
python3 scripts/train_chunk.py --start 0      --count 150000            # chunk 1 (fresh build)
python3 scripts/train_chunk.py --start 150000 --count 150000            # ... continue to 600k
python3 scripts/train_chunk.py --start 0      --count 150000            # epoch 2
python3 scripts/train_chunk.py --start 450000 --count 150000 --raster   # epoch 2 end + raster
python3 scripts/train_chunk.py --am-only                               # hippocampal write pass (1.2M)
python3 scripts/train_chunk.py --start 450000 --warmup 150000           # re-adapt operating point

# full evaluation (resumable: completed sections are skipped on re-run)
python3 -m spira.evaluate --model artifacts/spira_model.npz \
    --train data/train.bin --test data/test.bin \
    --oneshot data/oneshot.bin --out artifacts/metrics.json

# hippocampal replay -> cortical consolidation (in a few lines)
python3 - << 'EOF'
import sys; sys.path.insert(0, ".")
from spira import SPIRA
m = SPIRA.load("artifacts/spira_model.npz")
print(m.teach_bytes(open("data/oneshot.bin", "rb").read()[:60000]))
print("before replay:", m.eval_stream(open("data/oneshot.bin","rb").read()[:60000], use_am=False, max_positions=4000)["hier_acc"])
print(m.replay(100000))   # the "sleep" pass
print("after replay:", m.eval_stream(open("data/oneshot.bin","rb").read()[:60000], use_am=False, max_positions=4000)["hier_acc"])
EOF

# export the browser-demo artifact (format v2, includes the slow area)
python3 -m spira.export_deploy --model artifacts/spira_model.npz \
    --out artifacts/spira_deploy.json --am-cap 100000 --am-min-count 1
```

## Design notes (the interesting parts)

- **Constant-information tokens.** Each EPT patch carries ≈15.9 bits of accumulated
  surprisal; low-entropy text yields long patches, high-entropy text short ones.
  Patch identity is the FNV-1a-64 hash of the bytes — the same substring is always
  the same token, anywhere, with zero vocabulary learning.
- **The readout rule is local.** Readout units apply a Widrow–Hoff / Rescorla–Wagner
  delta rule on their own activity, gated by dopamine `d = r − baseline(r)`. No
  gradients, no chain rule, no autodiff.
- **Mean-centering the spike trace matters.** The raw trace carries a dense shared
  background that otherwise makes every column's score correlate with frequency;
  subtracting a slow EMA of the trace (homeostatic scaling) is what makes per-context
  learning converge.
- **Learned state includes the operating point.** Homeostatic thresholds, the trace
  mean, the slow area's homeostatic scalars, the top-down gain β, and STDP-shaped
  recurrent weights must be checkpointed together — omitting any of them (or
  resetting β to its init) silently destroys the readout's regime.
- **The hierarchy must be variance-matched and counterfactually credited.** The slow
  readout's raw scores live at ~20× the fast one's scale; fusing raw scores lets a
  tiny gain swamp the fast argmax, so the prior is rescaled by σ_fast/σ_slow
  (measured once, like the EPT budget). And a gain rule that rewards the slow area
  for ranking frequent truths high degenerates into a frequency prior — only
  saved-vs-corrupted counterfactual attribution calibrates β honestly (it
  self-discounts to ≈0.06 at this scale).
- **Hippocampal replay closes the complementary-systems loop.** Teaching retains the
  episodic token trace; replay re-runs random 512-token slices through the cortex in
  full training mode, so one-shot material becomes slow cortical weight while the
  hippocampus stays intact (0.3% → 46.9% cortex-only recall, zero interference —
  held-out accuracy actually improves slightly).
- **Associative-memory checkpoints are easy to get subtly wrong.** Entries must be
  packed and restored at their derivable slots (primary + leave-one-out hashes); an
  early format that rebuilt entries at lane-value addresses made the reloaded corpus
  memory invisible to queries — and inflated v1's corruption-robustness number,
  which had been measured against an accidentally-empty hippocampus.
- **Hybrid hippocampal addressing.** A primary full-key hash gives every distinct
  context its own slot (capacity); leave-one-out key hashes let a query with one
  corrupted lane still reach the entry (completion). Completion strength trades
  against table load (99% at 0.8% load vs 52% at 46% load); two corrupted lanes is
  an explicit architectural cliff.
- **The TS port is compatible.** `web/src/lib/spira/` regenerates the reservoir from
  the same seeded PRNG (mulberry32), shares the exact hashing spec, ports the slow
  area (stepG + variance-matched fusion), and loads the same deploy artifact, so
  browser inference matches the Python prototype. (Replay is Python-only.)

## Citation

See `../paper/spira_paper.pdf` — *SPIRA: Sequence Learning Without Backpropagation or
GPU Clusters*, preprint, October 2026 (third iteration).