File size: 7,028 Bytes
032a961 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: 'Helvetica Neue', Arial, sans-serif; background: #ffffff; width: 1180px; padding: 24px; }
.stage { display: flex; align-items: stretch; gap: 18px; }
.col { display: flex; flex-direction: column; gap: 14px; flex: 1; }
.box { border-radius: 10px; padding: 14px 16px; border: 1.5px solid; position: relative; }
.box h3 { font-size: 15px; margin-bottom: 6px; }
.box p { font-size: 11.5px; line-height: 1.45; color: #374151; }
.tag { position: absolute; top: -9px; left: 14px; font-size: 9.5px; letter-spacing: 1.5px;
background: #ffffff; padding: 0 6px; font-weight: 700; }
.ept { border-color: #2e7d5b; background: #effaf4; }
.ept .tag { color: #2e7d5b; }
.snn { border-color: #b8860b; background: #fdf8ec; }
.snn .tag { color: #b8860b; }
.slow { border-color: #c2701e; background: #fdf3e7; border-style: dashed; }
.slow .tag { color: #c2701e; }
.am { border-color: #8b4a6b; background: #faf1f5; }
.am .tag { color: #8b4a6b; }
.rep { border-color: #8b4a6b; background: #faf1f5; border-style: dashed; }
.rep .tag { color: #8b4a6b; }
.io { border-color: #4b5563; background: #f4f5f6; }
.io .tag { color: #4b5563; }
.flow { display: flex; flex-direction: column; justify-content: space-around; align-items: center; }
.arrow { display: flex; flex-direction: column; align-items: center; gap: 2px; }
.arrow .line { width: 2px; height: 34px; background: #9ca3af; position: relative; }
.arrow .head { width: 0; height: 0; border-left: 6px solid transparent; border-right: 6px solid transparent; border-top: 8px solid #9ca3af; }
.arrow .lbl { font-size: 9.5px; color: #6b7280; margin: 2px 0; text-align: center; }
.loop { font-size: 10px; color: #8b4a6b; text-align: center; margin-top: 6px; line-height: 1.5; }
.metrics { margin-top: 20px; display: flex; gap: 14px; }
.metric { flex: 1; border: 1px solid #e5e7eb; border-radius: 8px; padding: 10px 12px; background: #fafafa; }
.metric .v { font-size: 17px; font-weight: 700; color: #1f2937; }
.metric .k { font-size: 10px; color: #6b7280; margin-top: 2px; line-height: 1.35; }
.accent { color: #2e7d5b; } .accent2 { color: #b8860b; } .accent3 { color: #8b4a6b; } .accent4 { color: #c2701e; }
</style>
</head>
<body>
<div class="stage">
<div class="col" style="flex: 0 0 200px;">
<div class="box io">
<span class="tag">INPUT</span>
<h3>Raw byte stream</h3>
<p>UTF-8 text + source code, 13.2 MB mixed corpus. No vocabulary, no pre-training, no GPU.</p>
</div>
<div class="box ept">
<span class="tag">TOKENIZER</span>
<h3>EPT — Entropy Patches</h3>
<p>Order-1 byte Markov surprisal; greedy segmentation where each patch carries ≈15.9 bits of accumulated information. Content-addressed by FNV-1a-64: the same substring is always the same token.</p>
</div>
</div>
<div class="flow">
<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">patches</div></div>
</div>
<div class="col" style="flex: 0 0 300px;">
<div class="box io">
<span class="tag">PROJECTION</span>
<h3>Hashing trick</h3>
<p>Each patch id projects to 12 of 1024 input neurons via splitmix64 — no embedding table, works for unseen patches.</p>
</div>
<div class="box snn">
<span class="tag">CORTEX · FAST</span>
<h3>Spiking reservoir (L)</h3>
<p>512 LIF neurons, Dale's law (50% inhibitory), homeostatic thresholds, refractory periods. Recurrent synapses follow reward-modulated STDP.</p>
</div>
<div class="box slow">
<span class="tag">CORTEX · SLOW</span>
<h3>Slow area (G) — hierarchy</h3>
<p>256 LIF neurons reading the pooled fast spike volley every 8 tokens (multi-scale). Its readout contributes a top-down prior fused with gain β (variance-matched). A counterfactual trust rule self-calibrates β.</p>
</div>
<div class="box snn">
<span class="tag">READOUT</span>
<h3>Three-factor readout</h3>
<p>Local Rescorla–Wagner delta rule on active units, gated by a dopaminergic RPE broadcast to both areas. No backprop, no gradients, no autodiff.</p>
</div>
</div>
<div class="flow">
<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">scores</div></div>
</div>
<div class="col" style="flex: 0 0 260px;">
<div class="box am">
<span class="tag">HIPPOCAMPUS</span>
<h3>One-shot memory</h3>
<p>Hybrid addressing: primary full-key hash (pattern separation) + leave-one-out lanes (CA3-style completion). Writes associations in a single pass and keeps the episodic trace.</p>
</div>
<div class="box rep">
<span class="tag">REPLAY ↺</span>
<h3>Hippocampal → cortex</h3>
<p>Offline "sleep": episodic traces are re-run through the cortex in training mode, consolidating one-shot memories into slow cortical weights. Cortex-only recall rises 0.3% → 46.9% while the hippocampus stays intact.</p>
<div class="loop">↺ closes the complementary-systems loop</div>
</div>
<div class="box io">
<span class="tag">FUSION</span>
<h3>Episodic + statistical</h3>
<p>Readout scores fused with hippocampal votes; strong recall overrides the vocabulary. Closed loop: the emitted patch is fed back as spikes.</p>
</div>
</div>
<div class="flow">
<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">next patch</div></div>
</div>
<div class="col" style="flex: 0 0 160px;">
<div class="box io" style="flex: 1; display:flex; flex-direction:column; justify-content:center;">
<span class="tag">OUTPUT</span>
<h3>Generation</h3>
<p>Multi-byte patches emitted at ≈2,000 chars/s on 2 vCPU. One-shot teaching of new text in 0.4 s. Readout vocabulary 16,384 (64% held-out coverage).</p>
</div>
</div>
</div>
<div class="metrics">
<div class="metric"><div class="v accent">99.3%</div><div class="k">one-shot recall after a single pass on an unseen book</div></div>
<div class="metric"><div class="v accent4">0.3→46.9%</div><div class="k">cortex-only recall of the book after hippocampal replay (138×; in-vocab 64.2%)</div></div>
<div class="metric"><div class="v accent2">99.0% / 52%</div><div class="k">recall with 1-of-4 corrupted context — sparse vs full hippocampus (capacity/completion trade)</div></div>
<div class="metric"><div class="v">1.2M</div><div class="k">associations written in 18 s (pure CPU), exact checkpoint round-trip</div></div>
<div class="metric"><div class="v accent3">9.2M</div><div class="k">synaptic MACs per token — ≈81× fewer ops/token than GPT-2-small training</div></div>
</div>
</body>
</html>
|