python/README.md

SPIRA — Spiking Plastic Recurrent Architecture

A CPU-only sequence-learning system outside the transformer/BPE/backpropagation paradigm, built from three components:

Component	What it replaces
EPT — Entropy-Patch Tokenization	BPE / WordPiece / SentencePiece
Hierarchical spiking cortex — LIF reservoir + slow area, reward-modulated plasticity	Transformer + backprop
Hippocampus — one-shot associative memory + replay consolidation	Fine-tuning / gradient learning

This is v2 of the prototype: a 16,384-column readout (chosen from a measured coverage curve), a two-timescale hierarchical cortex (fast 512-neuron reservoir + slow 256-neuron area reading the pooled spike volley every 8 tokens), and hippocampal replay that consolidates one-shot memories into cortical weights.

Headline results (2 vCPU, 4 GB RAM, 13.2 MB mixed text+code corpus)

Proof	Result
One-shot learning	99.3% exact recall after a single 0.4 s pass over an unseen book
Systems consolidation (v2)	cortex-only recall of that book 0.3% → 46.9% (64.2% in-vocab, 138×) after replaying 100k hippocampal tokens — the AM itself stays at 99.3%
Pattern completion	99.0% recall with 1 of 4 context patches corrupted (sparse AM); 52.3% in the deployed full AM (46% load)
Replay interference	none — held-out test accuracy improves slightly after replay (0.34% → 0.75% cortex-only)
Zero-shot coverage	6.3% patch novelty on held-out text+code; the 16k readout covers 64% of held-out targets (v1: 38% at 4k)
Efficiency	≈9.2 M synaptic MACs/token — ≈81× fewer ops/token than GPT-2-small training (v1 at 4k vocab: 2.4 M, 314×)
Generation	≈2,000 chars/s closed-loop on 2 vCPU, in-browser at ~450-700 chars/s
One-shot write rate	1,199,996 associations in 18 s, with an exact checkpoint round-trip

Honest negative results (reported in the paper): the cortex does not amortize generalization to unseen documents (test in-vocab 0.49% vs a bigram's 7.7%), and the slow area's token-level fusion self-discounts to β≈0.06 at this scale — its value is as a second consolidation substrate during replay. See ../paper/spira_paper.pdf, Sections 7–8.

What's new in v3 (cortex representation repair)

Diagnosis probes on the shipped v2 checkpoint showed the held-out weakness was representational: the readout state carried ≈1 token of memory (current patch decodable at 99.94%, previous patch at 39.7% vs 33.0% majority chance), and an offline ridge readout on the same states scored 0.36% — worse than the online delta readout (0.51%). v3 changes the readout's input, not its learning rule:

• Direct input pathway (direct_input=True, direct_tau=0.5): a decaying trace of the exact input-neuron pattern is concatenated to the readout state, so the delta rule sees the recent input identities directly.
• Multi-timescale trace banks (trace_taus=(0.4, 0.8, 0.95)): the reservoir keeps one spike-trace bank per τ; slow banks are normalized by (1 − τ) so the τ=0.95 bank does not swamp the fast one. With the direct pathway, the readout state is n_state = 512 × 3 + 1024 = 2560 (v2: 512), and W_out is (2560, V+1).
• Structured inputs (structured_inputs=True, k_tri=6): each patch activates 6 full-hash input neurons plus 6 byte-trigram hash neurons, so patches sharing byte trigrams share input neurons.
• Compatibility mode: direct_input=False, trace_taus=(0.4,), structured_inputs=False reproduces the v2 architecture exactly (n_state = 512); v3 checkpoints record the new fields and restore them on SPIRA.load().
• Bug fix: v2's model.train() referenced an undefined ok_mask (NameError) — training only ever ran through scripts/train_chunk.py. Fixed in v3; SPIRA.train() now runs standalone.

Controlled A/B (same protocol both arms — rebuilt corpus, vocab 4096, 200k tokens × 2 epochs, AM 50k contexts, held-out test 20k positions, coverage 45.24%): SNN alone, in-vocab — 0.40% (v2) → 2.07% (v3), patch bigram @4096 on the same positions 4.94%. The beat-the-bigram criterion is not met (gap ×12.4 → ×2.4); trained region 3.76% → 13.29%. Training costs 189 t/s vs ~530 t/s and the checkpoint grows 14.6 MB → 45.9 MB. See ../V3_REPORT.md and artifacts/metrics_v3_ab.json.

Repository layout

spira/                  Python package (NumPy only, CPU-only)
  hashing.py            FNV-1a-64, splitmix64, mulberry32 — bit-compatible with the TS port
  ept.py                entropy-patch tokenizer (order-1 Markov surprisal, content addressing)
  spiking.py            LIF reservoir + Dale's law + homeostatic thresholds,
                        Rescorla-Wagner readout gated by dopaminergic RPE,
                        reward-modulated recurrent STDP, and the hierarchical
                        slow area G (windowed spike volleys, top-down prior,
                        counterfactual trust gain beta); v3: multi-timescale
                        trace banks + direct input pathway to the readout
  hippocampus.py        one-shot memory: primary full-key hash (pattern separation)
                        + leave-one-out lanes (CA3 pattern completion), exact
                        derivable-slot checkpoint round-trip
  model.py              SPIRA wrapper: build / train / generate / teach / replay /
                        calibrate_fusion / eval / save-load; v3 config:
                        direct_input, trace_taus, direct_tau, structured_inputs,
                        k_tri (defaults reproduce v3; flags off = v2 architecture)
  baselines.py          gzip, byte-level BPE (HuggingFace tokenizers), sparse
                        n-gram predictors
  train.py              training CLI
  evaluate.py           full evaluation harness (coverage curve, compression,
                        prediction + hierarchy ablation, one-shot, REPLAY
                        consolidation experiment, correction, generation) —
                        resumable by section
  export_deploy.py      JSON artifact export for the browser demo (format v2)
artifacts/              trained checkpoint, metrics.json, deploy artifact,
                        figures, browser-demo screenshots
paper/                  LaTeX source + compiled research paper (spira_paper.pdf)

Quick start

# build the corpus (Gutenberg books + Python stdlib code)
python3 scripts/build_corpus.py

# chunked training (background processes are killed between calls in sandboxes;
# each chunk fits a single tool/shell call). v2 defaults: vocab 16384, hierarchy on.
python3 scripts/train_chunk.py --start 0      --count 150000            # chunk 1 (fresh build)
python3 scripts/train_chunk.py --start 150000 --count 150000            # ... continue to 600k
python3 scripts/train_chunk.py --start 0      --count 150000            # epoch 2
python3 scripts/train_chunk.py --start 450000 --count 150000 --raster   # epoch 2 end + raster
python3 scripts/train_chunk.py --am-only                               # hippocampal write pass (1.2M)
python3 scripts/train_chunk.py --start 450000 --warmup 150000           # re-adapt operating point

# full evaluation (resumable: completed sections are skipped on re-run)
python3 -m spira.evaluate --model artifacts/spira_model.npz \
    --train data/train.bin --test data/test.bin \
    --oneshot data/oneshot.bin --out artifacts/metrics.json

# hippocampal replay -> cortical consolidation (in a few lines)
python3 - << 'EOF'
import sys; sys.path.insert(0, ".")
from spira import SPIRA
m = SPIRA.load("artifacts/spira_model.npz")
print(m.teach_bytes(open("data/oneshot.bin", "rb").read()[:60000]))
print("before replay:", m.eval_stream(open("data/oneshot.bin","rb").read()[:60000], use_am=False, max_positions=4000)["hier_acc"])
print(m.replay(100000))   # the "sleep" pass
print("after replay:", m.eval_stream(open("data/oneshot.bin","rb").read()[:60000], use_am=False, max_positions=4000)["hier_acc"])
EOF

# export the browser-demo artifact (format v2, includes the slow area)
python3 -m spira.export_deploy --model artifacts/spira_model.npz \
    --out artifacts/spira_deploy.json --am-cap 100000 --am-min-count 1

Design notes (the interesting parts)

• Constant-information tokens. Each EPT patch carries ≈15.9 bits of accumulated surprisal; low-entropy text yields long patches, high-entropy text short ones. Patch identity is the FNV-1a-64 hash of the bytes — the same substring is always the same token, anywhere, with zero vocabulary learning.
• The readout rule is local. Readout units apply a Widrow–Hoff / Rescorla–Wagner delta rule on their own activity, gated by dopamine d = r − baseline(r). No gradients, no chain rule, no autodiff.
• Mean-centering the spike trace matters. The raw trace carries a dense shared background that otherwise makes every column's score correlate with frequency; subtracting a slow EMA of the trace (homeostatic scaling) is what makes per-context learning converge.
• Learned state includes the operating point. Homeostatic thresholds, the trace mean, the slow area's homeostatic scalars, the top-down gain β, and STDP-shaped recurrent weights must be checkpointed together — omitting any of them (or resetting β to its init) silently destroys the readout's regime.
• The hierarchy must be variance-matched and counterfactually credited. The slow readout's raw scores live at ~20× the fast one's scale; fusing raw scores lets a tiny gain swamp the fast argmax, so the prior is rescaled by σ_fast/σ_slow (measured once, like the EPT budget). And a gain rule that rewards the slow area for ranking frequent truths high degenerates into a frequency prior — only saved-vs-corrupted counterfactual attribution calibrates β honestly (it self-discounts to ≈0.06 at this scale).
• Hippocampal replay closes the complementary-systems loop. Teaching retains the episodic token trace; replay re-runs random 512-token slices through the cortex in full training mode, so one-shot material becomes slow cortical weight while the hippocampus stays intact (0.3% → 46.9% cortex-only recall, zero interference — held-out accuracy actually improves slightly).
• Associative-memory checkpoints are easy to get subtly wrong. Entries must be packed and restored at their derivable slots (primary + leave-one-out hashes); an early format that rebuilt entries at lane-value addresses made the reloaded corpus memory invisible to queries — and inflated v1's corruption-robustness number, which had been measured against an accidentally-empty hippocampus.
• Hybrid hippocampal addressing. A primary full-key hash gives every distinct context its own slot (capacity); leave-one-out key hashes let a query with one corrupted lane still reach the entry (completion). Completion strength trades against table load (99% at 0.8% load vs 52% at 46% load); two corrupted lanes is an explicit architectural cliff.
• The TS port is compatible. web/src/lib/spira/ regenerates the reservoir from the same seeded PRNG (mulberry32), shares the exact hashing spec, ports the slow area (stepG + variance-matched fusion), and loads the same deploy artifact, so browser inference matches the Python prototype. (Replay is Python-only.)

Citation

See ../paper/spira_paper.pdf — SPIRA: Sequence Learning Without Backpropagation or GPU Clusters, preprint, October 2026 (third iteration).