SPIRA / python /scripts /fig_architecture.html

SPIRA v2: entropy-patch tokenization + hierarchical dopamine-gated spiking cortex + hippocampal one-shot memory with replay consolidation

032a961 verified 18 days ago

Raw History Blame Contribute Delete

7.03 kB

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<style>
  * { box-sizing: border-box; margin: 0; padding: 0; }
  body { font-family: 'Helvetica Neue', Arial, sans-serif; background: #ffffff; width: 1180px; padding: 24px; }
  .stage { display: flex; align-items: stretch; gap: 18px; }
  .col { display: flex; flex-direction: column; gap: 14px; flex: 1; }
  .box { border-radius: 10px; padding: 14px 16px; border: 1.5px solid; position: relative; }
  .box h3 { font-size: 15px; margin-bottom: 6px; }
  .box p { font-size: 11.5px; line-height: 1.45; color: #374151; }
  .tag { position: absolute; top: -9px; left: 14px; font-size: 9.5px; letter-spacing: 1.5px;
         background: #ffffff; padding: 0 6px; font-weight: 700; }
  .ept { border-color: #2e7d5b; background: #effaf4; }
  .ept .tag { color: #2e7d5b; }
  .snn { border-color: #b8860b; background: #fdf8ec; }
  .snn .tag { color: #b8860b; }
  .slow { border-color: #c2701e; background: #fdf3e7; border-style: dashed; }
  .slow .tag { color: #c2701e; }
  .am  { border-color: #8b4a6b; background: #faf1f5; }
  .am .tag { color: #8b4a6b; }
  .rep { border-color: #8b4a6b; background: #faf1f5; border-style: dashed; }
  .rep .tag { color: #8b4a6b; }
  .io  { border-color: #4b5563; background: #f4f5f6; }
  .io .tag { color: #4b5563; }
  .flow { display: flex; flex-direction: column; justify-content: space-around; align-items: center; }
  .arrow { display: flex; flex-direction: column; align-items: center; gap: 2px; }
  .arrow .line { width: 2px; height: 34px; background: #9ca3af; position: relative; }
  .arrow .head { width: 0; height: 0; border-left: 6px solid transparent; border-right: 6px solid transparent; border-top: 8px solid #9ca3af; }
  .arrow .lbl { font-size: 9.5px; color: #6b7280; margin: 2px 0; text-align: center; }
  .loop { font-size: 10px; color: #8b4a6b; text-align: center; margin-top: 6px; line-height: 1.5; }
  .metrics { margin-top: 20px; display: flex; gap: 14px; }
  .metric { flex: 1; border: 1px solid #e5e7eb; border-radius: 8px; padding: 10px 12px; background: #fafafa; }
  .metric .v { font-size: 17px; font-weight: 700; color: #1f2937; }
  .metric .k { font-size: 10px; color: #6b7280; margin-top: 2px; line-height: 1.35; }
  .accent { color: #2e7d5b; } .accent2 { color: #b8860b; } .accent3 { color: #8b4a6b; } .accent4 { color: #c2701e; }
</style>
</head>
<body>
  <div class="stage">
    <div class="col" style="flex: 0 0 200px;">
      <div class="box io">
        <span class="tag">INPUT</span>
        <h3>Raw byte stream</h3>
        <p>UTF-8 text + source code, 13.2&nbsp;MB mixed corpus. No vocabulary, no pre-training, no GPU.</p>
      </div>
      <div class="box ept">
        <span class="tag">TOKENIZER</span>
        <h3>EPT — Entropy Patches</h3>
        <p>Order-1 byte Markov surprisal; greedy segmentation where each patch carries ≈15.9 bits of accumulated information. Content-addressed by FNV-1a-64: the same substring is always the same token.</p>
      </div>
    </div>
    <div class="flow">
      <div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">patches</div></div>
    </div>
    <div class="col" style="flex: 0 0 300px;">
      <div class="box io">
        <span class="tag">PROJECTION</span>
        <h3>Hashing trick</h3>
        <p>Each patch id projects to 12 of 1024 input neurons via splitmix64 — no embedding table, works for unseen patches.</p>
      </div>
      <div class="box snn">
        <span class="tag">CORTEX · FAST</span>
        <h3>Spiking reservoir (L)</h3>
        <p>512 LIF neurons, Dale's law (50% inhibitory), homeostatic thresholds, refractory periods. Recurrent synapses follow reward-modulated STDP.</p>
      </div>
      <div class="box slow">
        <span class="tag">CORTEX · SLOW</span>
        <h3>Slow area (G) — hierarchy</h3>
        <p>256 LIF neurons reading the pooled fast spike volley every 8 tokens (multi-scale). Its readout contributes a top-down prior fused with gain&nbsp;β (variance-matched). A counterfactual trust rule self-calibrates β.</p>
      </div>
      <div class="box snn">
        <span class="tag">READOUT</span>
        <h3>Three-factor readout</h3>
        <p>Local Rescorla–Wagner delta rule on active units, gated by a dopaminergic RPE broadcast to both areas. No backprop, no gradients, no autodiff.</p>
      </div>
    </div>
    <div class="flow">
      <div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">scores</div></div>
    </div>
    <div class="col" style="flex: 0 0 260px;">
      <div class="box am">
        <span class="tag">HIPPOCAMPUS</span>
        <h3>One-shot memory</h3>
        <p>Hybrid addressing: primary full-key hash (pattern separation) + leave-one-out lanes (CA3-style completion). Writes associations in a single pass and keeps the episodic trace.</p>
      </div>
      <div class="box rep">
        <span class="tag">REPLAY ↺</span>
        <h3>Hippocampal → cortex</h3>
        <p>Offline "sleep": episodic traces are re-run through the cortex in training mode, consolidating one-shot memories into slow cortical weights. Cortex-only recall rises 0.3%&nbsp;→&nbsp;46.9% while the hippocampus stays intact.</p>
        <div class="loop">↺ closes the complementary-systems loop</div>
      </div>
      <div class="box io">
        <span class="tag">FUSION</span>
        <h3>Episodic + statistical</h3>
        <p>Readout scores fused with hippocampal votes; strong recall overrides the vocabulary. Closed loop: the emitted patch is fed back as spikes.</p>
      </div>
    </div>
    <div class="flow">
      <div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">next patch</div></div>
    </div>
    <div class="col" style="flex: 0 0 160px;">
      <div class="box io" style="flex: 1; display:flex; flex-direction:column; justify-content:center;">
        <span class="tag">OUTPUT</span>
        <h3>Generation</h3>
        <p>Multi-byte patches emitted at ≈2,000 chars/s on 2 vCPU. One-shot teaching of new text in 0.4&nbsp;s. Readout vocabulary 16,384 (64% held-out coverage).</p>
      </div>
    </div>
  </div>
  <div class="metrics">
    <div class="metric"><div class="v accent">99.3%</div><div class="k">one-shot recall after a single pass on an unseen book</div></div>
    <div class="metric"><div class="v accent4">0.3→46.9%</div><div class="k">cortex-only recall of the book after hippocampal replay (138×; in-vocab 64.2%)</div></div>
    <div class="metric"><div class="v accent2">99.0% / 52%</div><div class="k">recall with 1-of-4 corrupted context — sparse vs full hippocampus (capacity/completion trade)</div></div>
    <div class="metric"><div class="v">1.2M</div><div class="k">associations written in 18 s (pure CPU), exact checkpoint round-trip</div></div>
    <div class="metric"><div class="v accent3">9.2M</div><div class="k">synaptic MACs per token — ≈81× fewer ops/token than GPT-2-small training</div></div>
  </div>
</body>
</html>

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<style>
	* { box-sizing: border-box; margin: 0; padding: 0; }
	body { font-family: 'Helvetica Neue', Arial, sans-serif; background: #ffffff; width: 1180px; padding: 24px; }
	.stage { display: flex; align-items: stretch; gap: 18px; }
	.col { display: flex; flex-direction: column; gap: 14px; flex: 1; }
	.box { border-radius: 10px; padding: 14px 16px; border: 1.5px solid; position: relative; }
	.box h3 { font-size: 15px; margin-bottom: 6px; }
	.box p { font-size: 11.5px; line-height: 1.45; color: #374151; }
	.tag { position: absolute; top: -9px; left: 14px; font-size: 9.5px; letter-spacing: 1.5px;
	background: #ffffff; padding: 0 6px; font-weight: 700; }
	.ept { border-color: #2e7d5b; background: #effaf4; }
	.ept .tag { color: #2e7d5b; }
	.snn { border-color: #b8860b; background: #fdf8ec; }
	.snn .tag { color: #b8860b; }
	.slow { border-color: #c2701e; background: #fdf3e7; border-style: dashed; }
	.slow .tag { color: #c2701e; }
	.am { border-color: #8b4a6b; background: #faf1f5; }
	.am .tag { color: #8b4a6b; }
	.rep { border-color: #8b4a6b; background: #faf1f5; border-style: dashed; }
	.rep .tag { color: #8b4a6b; }
	.io { border-color: #4b5563; background: #f4f5f6; }
	.io .tag { color: #4b5563; }
	.flow { display: flex; flex-direction: column; justify-content: space-around; align-items: center; }
	.arrow { display: flex; flex-direction: column; align-items: center; gap: 2px; }
	.arrow .line { width: 2px; height: 34px; background: #9ca3af; position: relative; }
	.arrow .head { width: 0; height: 0; border-left: 6px solid transparent; border-right: 6px solid transparent; border-top: 8px solid #9ca3af; }
	.arrow .lbl { font-size: 9.5px; color: #6b7280; margin: 2px 0; text-align: center; }
	.loop { font-size: 10px; color: #8b4a6b; text-align: center; margin-top: 6px; line-height: 1.5; }
	.metrics { margin-top: 20px; display: flex; gap: 14px; }
	.metric { flex: 1; border: 1px solid #e5e7eb; border-radius: 8px; padding: 10px 12px; background: #fafafa; }
	.metric .v { font-size: 17px; font-weight: 700; color: #1f2937; }
	.metric .k { font-size: 10px; color: #6b7280; margin-top: 2px; line-height: 1.35; }
	.accent { color: #2e7d5b; } .accent2 { color: #b8860b; } .accent3 { color: #8b4a6b; } .accent4 { color: #c2701e; }
	</style>
	</head>
	<body>
	<div class="stage">
	<div class="col" style="flex: 0 0 200px;">
	<div class="box io">
	<span class="tag">INPUT</span>
	<h3>Raw byte stream</h3>
	<p>UTF-8 text + source code, 13.2 MB mixed corpus. No vocabulary, no pre-training, no GPU.</p>
	</div>
	<div class="box ept">
	<span class="tag">TOKENIZER</span>
	<h3>EPT — Entropy Patches</h3>
	<p>Order-1 byte Markov surprisal; greedy segmentation where each patch carries ≈15.9 bits of accumulated information. Content-addressed by FNV-1a-64: the same substring is always the same token.</p>
	</div>
	</div>
	<div class="flow">
	<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">patches</div></div>
	</div>
	<div class="col" style="flex: 0 0 300px;">
	<div class="box io">
	<span class="tag">PROJECTION</span>
	<h3>Hashing trick</h3>
	<p>Each patch id projects to 12 of 1024 input neurons via splitmix64 — no embedding table, works for unseen patches.</p>
	</div>
	<div class="box snn">
	<span class="tag">CORTEX · FAST</span>
	<h3>Spiking reservoir (L)</h3>
	<p>512 LIF neurons, Dale's law (50% inhibitory), homeostatic thresholds, refractory periods. Recurrent synapses follow reward-modulated STDP.</p>
	</div>
	<div class="box slow">
	<span class="tag">CORTEX · SLOW</span>
	<h3>Slow area (G) — hierarchy</h3>
	<p>256 LIF neurons reading the pooled fast spike volley every 8 tokens (multi-scale). Its readout contributes a top-down prior fused with gain β (variance-matched). A counterfactual trust rule self-calibrates β.</p>
	</div>
	<div class="box snn">
	<span class="tag">READOUT</span>
	<h3>Three-factor readout</h3>
	<p>Local Rescorla–Wagner delta rule on active units, gated by a dopaminergic RPE broadcast to both areas. No backprop, no gradients, no autodiff.</p>
	</div>
	</div>
	<div class="flow">
	<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">scores</div></div>
	</div>
	<div class="col" style="flex: 0 0 260px;">
	<div class="box am">
	<span class="tag">HIPPOCAMPUS</span>
	<h3>One-shot memory</h3>
	<p>Hybrid addressing: primary full-key hash (pattern separation) + leave-one-out lanes (CA3-style completion). Writes associations in a single pass and keeps the episodic trace.</p>
	</div>
	<div class="box rep">
	<span class="tag">REPLAY ↺</span>
	<h3>Hippocampal → cortex</h3>
	<p>Offline "sleep": episodic traces are re-run through the cortex in training mode, consolidating one-shot memories into slow cortical weights. Cortex-only recall rises 0.3% → 46.9% while the hippocampus stays intact.</p>
	<div class="loop">↺ closes the complementary-systems loop</div>
	</div>
	<div class="box io">
	<span class="tag">FUSION</span>
	<h3>Episodic + statistical</h3>
	<p>Readout scores fused with hippocampal votes; strong recall overrides the vocabulary. Closed loop: the emitted patch is fed back as spikes.</p>
	</div>
	</div>
	<div class="flow">
	<div class="arrow"><div class="line"></div><div class="head"></div><div class="lbl">next patch</div></div>
	</div>
	<div class="col" style="flex: 0 0 160px;">
	<div class="box io" style="flex: 1; display:flex; flex-direction:column; justify-content:center;">
	<span class="tag">OUTPUT</span>
	<h3>Generation</h3>
	<p>Multi-byte patches emitted at ≈2,000 chars/s on 2 vCPU. One-shot teaching of new text in 0.4 s. Readout vocabulary 16,384 (64% held-out coverage).</p>
	</div>
	</div>
	</div>
	<div class="metrics">
	<div class="metric"><div class="v accent">99.3%</div><div class="k">one-shot recall after a single pass on an unseen book</div></div>
	<div class="metric"><div class="v accent4">0.3→46.9%</div><div class="k">cortex-only recall of the book after hippocampal replay (138×; in-vocab 64.2%)</div></div>
	<div class="metric"><div class="v accent2">99.0% / 52%</div><div class="k">recall with 1-of-4 corrupted context — sparse vs full hippocampus (capacity/completion trade)</div></div>
	<div class="metric"><div class="v">1.2M</div><div class="k">associations written in 18 s (pure CPU), exact checkpoint round-trip</div></div>
	<div class="metric"><div class="v accent3">9.2M</div><div class="k">synaptic MACs per token — ≈81× fewer ops/token than GPT-2-small training</div></div>
	</div>
	</body>
	</html>