Promote 32-bit/64KB build to canonical, rewrite README

Root canonical neural_computer.safetensors is the most capable variant
(32-bit data path, 64 KB memory, ~8.47 M params). The legacy 8-bit/1KB
file at the same path and the redundant root neural_alu32.safetensors are
removed; equivalent configurations remain in variants/.

play.py is now manifest-aware and uses eval_all.py's GenericThresholdCPU,
defaulting to the small (1 KB) profile for fast interactive demos.

README rewritten as a self-contained overview: what the repo is, the file
layout, quick-start code, the architecture (ISA, state layout, instruction
encoding, circuit categories, naming), bit widths and memory profiles,
verification, threshold-logic background and history, neuromorphic
hardware compatibility, and the SmolLM2 LLM integration.

Files changed (4) hide show

README.md +480 -793
neural_alu32.safetensors +0 -3
neural_computer.safetensors +2 -2
play.py +217 -457

README.md CHANGED Viewed

@@ -1,793 +1,480 @@
----
-license: mit
-tags:
-- threshold-logic
-- neuromorphic
-- computer-architecture
-- turing-complete
-- loihi
-- truenorth
-- akida
----
-# 8bit-threshold-computer
-**A Turing-complete CPU implemented entirely as threshold logic gates, with 8-bit and 32-bit ALU support.**
-Every logic gate is a threshold neuron: `output = 1 if (Σ wᵢxᵢ + b) ≥ 0 else 0`
-```
-8-bit CPU:   8,290,134 params (full) / 32,397 params (pure ALU)
-32-bit ALU:  202,869 params (1KB scratch memory)
-```
----
-## What Is This?
-A complete processor where every operation—from Boolean logic to arithmetic to control flow—is implemented using only weighted sums and step functions. No traditional gates.
-| Component | 8-bit CPU | 32-bit ALU |
-|-----------|-----------|------------|
-| Registers | 4 × 8-bit | N/A (pure computation) |
-| Memory | 0B–64KB configurable | 1KB scratch |
-| ALU | 16 ops @ 8-bit | ADD, SUB, MUL, DIV, CMP, bitwise, shifts |
-| Precision | 0–255 | 0–4,294,967,295 |
-| Flags | Z, N, C, V | Carry/overflow |
-| Control | Full ISA | Stateless |
-**Turing complete.** The 8-bit CPU is verified with loops, conditionals, recursion, and self-modification. The 32-bit ALU extends arithmetic to practical ranges (0–4B) where 8-bit (0–255) is insufficient.
----
-## Execution Model
-A self-contained, autonomous computational machine:
-- **Pure tensor computation**: State in, state out
-- **Frozen circuits**: Integer weights, Heaviside activation
-- **ACT execution**: Internal loop until HALT
-- **No external orchestration**: One forward pass = complete program execution
-```
-            ┌─────────────────────────────┐
-            │      Initial State          │
-            │  [PC|Regs|Flags|Memory...]  │
-            └─────────────┬───────────────┘
-                          ▼
-            ┌─────────────────────────────┐
-            │                             │
-            │   Threshold Circuit Layer   │
-            │                             │
-            │  ┌───────────────────────┐  │
-            │  │   Fetch: PC → Instr   │  │
-            │  ├───────────────────────┤  │
-            │  │   Decode: Opcode/Ops  │  │
-            │  ├───────────────────────┤  │
-            │  │   Execute: ALU/Mem    │  │
-            │  ├───────────────────────┤  │
-            │  │   Writeback: Results  │  │
-            │  ├───────────────────────┤  │
-            │  │   PC Update           │  │
-            │  └───────────┬───────────┘  │
-            │              │              │
-            │         ┌────▼────┐         │
-            │         │ HALTED? │         │
-            │         └────┬────┘         │
-            │              │              │
-            │         no ──┴── yes        │
-            │          │       │          │
-            │          ▼       ▼          │
-            │       [loop]   [exit]       │
-            │                             │
-            └─────────────┬───────────────┘
-                          ▼
-            ┌─────────────────────────────┐
-            │       Final State           │
-            │  [PC|Regs|Flags|Memory...]  │
-            └─────────────────────────────┘
-```
-### Instruction Set
-| Opcode | Mnemonic | Operation |
-|--------|----------|-----------|
-| 0x0 | ADD | R[d] = R[a] + R[b] |
-| 0x1 | SUB | R[d] = R[a] - R[b] |
-| 0x2 | AND | R[d] = R[a] & R[b] |
-| 0x3 | OR | R[d] = R[a] \| R[b] |
-| 0x4 | XOR | R[d] = R[a] ^ R[b] |
-| 0x5 | SHL | R[d] = R[a] << 1 |
-| 0x6 | SHR | R[d] = R[a] >> 1 |
-| 0x7 | MUL | R[d] = R[a] * R[b] |
-| 0x8 | DIV | R[d] = R[a] / R[b] |
-| 0x9 | CMP | flags = R[a] - R[b] |
-| 0xA | LOAD | R[d] = M[addr] |
-| 0xB | STORE | M[addr] = R[s] |
-| 0xC | JMP | PC = addr |
-| 0xD | Jcc | PC = addr if cond (imm8[2:0]: 0=Z,1=NZ,2=C,3=NC,4=N,5=P,6=V,7=NV) |
-| 0xE | CALL | push PC; PC = addr |
-| 0xF | HALT | stop execution |
-### Design Principles
-1. **Autonomy**: Machine runs without external logic
-2. **Purity**: forward(state) → state', no side effects
-3. **Transparency**: All weights inspectable, all operations traceable
-4. **Universality**: Turing complete, runs arbitrary programs
----
-## Background
-### Threshold Logic
-A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing it to a threshold. If the sum meets or exceeds the threshold, the output is 1; otherwise, 0. This can be expressed as a neuron with Heaviside step activation: `output = 1 if (Σ wᵢxᵢ + b) ≥ 0 else 0`, where weights `wᵢ` and bias `b` are integers.
-Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function—this includes AND, OR, NAND, NOR, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (such as XOR or parity) require multiple threshold gates arranged in layers.
-### Historical Development
-Warren McCulloch and Walter Pitts introduced the threshold neuron model in 1943, proving that networks of such neurons could compute any Boolean function. This work preceded both the perceptron and modern neural networks, establishing the theoretical foundation for neural computation.
-The 1960s saw significant development in threshold logic synthesis. Researchers including Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function could be implemented as a single threshold gate, and if so, how to calculate appropriate weights. This work produced systematic techniques for threshold gate design but focused on individual gates rather than complete systems.
-Frank Rosenblatt's Mark I Perceptron (1957-1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training—the final weight configurations were not published. Bernard Widrow's ADALINE and MADALINE systems (1960-1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm.
-Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing complete. Their construction, however, relied on continuous sigmoid activation functions with infinite precision—not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: proving computational universality using continuous, differentiable activations suitable for gradient-based training.
-### Neuromorphic Hardware
-Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon:
-**Intel Loihi** (2017) provides 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. The architecture supports integer weights and configurable neuron dynamics.
-**IBM TrueNorth** (2014) integrates one million neurons and 256 million synapses in a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds. The chip was designed as an alternative to von Neumann architecture rather than an implementation of one.
-**BrainChip Akida** (2021) targets edge deployment with event-based processing and integer weights. The architecture supports standard neural network operations mapped onto neuromorphic primitives.
-**SpiNNaker** (University of Manchester) uses ARM processor cores to simulate spiking neural networks at scale. The platform has hosted various neural models but is simulation-based rather than native neuromorphic silicon.
-Despite the availability of these platforms, published work has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1, describing this as "a first step toward the construction of a spiking computer"—the implementation lacked instruction fetch, program counter, memory systems, and control logic.
-### This Implementation
-The weights in this repository implement a complete 8-bit computer: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every component is built from threshold neurons with integer weights. The weight configurations are published in safetensors format for direct loading and deployment.
----
-## Circuit Categories
-| Category | Circuits | Examples |
-|----------|----------|----------|
-| Boolean | 9 | AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES |
-| Arithmetic | 18 | Half/full adder, 2/4/8-bit ripple carry, comparators |
-| ALU | 3 | 8-bit ALU, control decoder, flag computation |
-| Combinational | 10 | MUX (2:1, 4:1, 8:1), DEMUX, encoders, decoders |
-| Control Flow | 16 | JMP, conditional jumps, CALL, RET, PUSH, POP |
-| Error Detection | 11 | Parity (XOR tree), checksum, CRC, Hamming |
-| Modular | 11 | Divisibility by 2-12 (multi-layer for non-powers-of-2) |
-| Threshold | 13 | k-of-n gates, majority, minority, exactly-k |
-| Pattern | 10 | Popcount, leading/trailing ones, symmetry |
-| Memory | 3 | N-bit addr decoder, 2^N×8 read mux, write cells (configurable, packed) |
----
-## Usage
-```python
-import torch
-from safetensors.torch import load_file
-tensors = load_file("neural_computer8.safetensors")
-def heaviside(x):
-    return (x >= 0).float()
-# AND gate: fires when both inputs are 1
-w = tensors['boolean.and.weight']  # [1, 1]
-b = tensors['boolean.and.bias']    # [-2]
-for a, b_in in [(0,0), (0,1), (1,0), (1,1)]:
-    inp = torch.tensor([a, b_in], dtype=torch.float32)
-    out = heaviside(inp @ w + b)
-    print(f"AND({a}, {b_in}) = {int(out.item())}")
-```
----
-## State Tensor Layout
-All multi-bit fields are **MSB-first** (index 0 is the most-significant bit).
-```
-[ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ]
-```
-Where N = address bits (configurable: 0-16).
-Flags are ordered as: `Z, N, C, V`.
-Control bits are ordered as: `HALT, MEM_WE, MEM_RE, RESERVED`.
-| Memory Profile | Addr Bits | Memory Size | State Bits |
-|----------------|-----------|-------------|------------|
-| Full CPU | 16 | 64KB | 524,376 |
-| Reduced | 12 | 4KB | 32,856 |
-| Scratchpad | 8 | 256B | 2,104 |
-| Registers | 4 | 16B | 184 |
-| Pure ALU | 0 | 0B | 56 |
----
-## Instruction Encoding (16-bit)
-All instruction bits are **MSB-first**.
-```
-15..12  11..10  9..8  7..0
-opcode  rd      rs    imm8
-```
-Interpretation:
-- **R-type**: `rd = rd op rs` (imm8 ignored).
-- **I-type**: `rd = op rd, imm8` (rs ignored).
-- **Address-extended**: `LOAD`, `STORE`, `JMP`, `JZ`, `CALL` consume the next word as a 16-bit address (big-endian). `imm8` is reserved, and the PC skips 4 bytes when the jump is not taken.
----
-## Verification
-```bash
-python eval.py
-python threshold_cpu.py
-```
-### Verification Status
-| Category | Status | Notes |
-|----------|--------|-------|
-| Boolean gates | Exhaustively tested | All 2^n input combinations |
-| Arithmetic | Exhaustively tested | Full 8-bit range |
-| ALU | Exhaustively tested | All operations, all inputs |
-| Control flow | Exhaustively tested | Branch/jump conditions |
-| Threshold | Exhaustively tested | k-of-n, majority, etc. |
-| Modular (mod 3,5,6,7,9,10,11,12) | Exhaustively tested | Multi-layer, hand-constructed |
-| Parity | Exhaustively tested | XOR tree, hand-constructed |
-| Modular (mod 2,4,8) | Exhaustively tested | Single-layer, trivial |
-The modular arithmetic circuits for non-powers-of-2 and the parity circuits were hand-constructed because:
-- Divisibility by 3, 5, etc. is **not linearly separable** in binary
-- 8-bit parity (XOR of all bits) requires a tree of XOR gates
-All circuits pass exhaustive testing over their full input domains.
----
-## Tensor Naming Convention
-```
-{category}.{circuit}[.{layer}][.{component}].{weight|bias}
-Examples:
-  boolean.and.weight
-  boolean.xor.layer1.neuron1.weight
-  arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight
-  modular.mod5.layer2.eq3.weight
-  error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias
-Memory circuits are stored as packed tensors to keep the safetensors header size manageable
-(e.g., `memory.addr_decode.weight`, `memory.read.and.weight`, `memory.write.and_old.weight`).
-```
----
-## Hardware Compatibility
-All weights are integers. All activations are Heaviside step. Designed for:
-- **Intel Loihi** — Neuromorphic research chip
-- **IBM TrueNorth** — 1M neuron chip
-- **BrainChip Akida** — Edge neuromorphic processor
----
-## LLM Integration
-The threshold circuits can be embedded into transformer MLP layers to give LLMs exact arithmetic capability.
-**For LLM integration, use `--memory-profile none` to generate a pure ALU model (~32K params) without memory circuits.**
-### Core Thesis
-Standard LLMs fail at arithmetic because they're interpolators—they approximate functions over training distributions rather than compute exact results. A 360M parameter model trained on internet text has seen "127 + 128 = 255" zero or few times, so it guesses based on pattern matching.
-We solve this by embedding **frozen, proven-correct arithmetic circuits** directly into the transformer's MLP layers. The circuits use threshold logic (weighted sums + step activation), which is structurally compatible with neural network layers. We train only the **interface layers** that learn to:
-1. Extract operands from token embeddings
-2. Route computation through the circuits
-3. Inject results back into the residual stream
-The model learns **call dispatch**, not arithmetic. The arithmetic is already solved.
-### Target Model: SmolLM2-360M-Instruct
-We use HuggingFace's SmolLM2-360M-Instruct as our base model. See [`llm_integration/SMOLLM2_ARCHITECTURE.md`](llm_integration/SMOLLM2_ARCHITECTURE.md) for the complete technical analysis.
-| Property | Value |
-|----------|-------|
-| Parameters | 361.82M |
-| Hidden Dimension | **960** (matches extractor input) |
-| Layers | 32 transformer blocks |
-| Attention | 15 query heads, 5 KV heads (GQA) |
-| MLP | SwiGLU (960→2560→960) |
-| Position Encoding | RoPE (theta=100k, max 8192) |
-**Key insight**: The hidden dimension of 960 exactly matches our extractor requirements—no projection layer needed.
-**Tokenization**: Digits are tokenized individually (`"47 + 86"` → `['4', '7', ' +', ' ', '8', '6']`), with digit token IDs following `token_id = 32 + digit_value`. This enables position-based operand extraction.
-**Hidden State Extraction**: Layer 31 (final, pre-LM-head) provides well-normalized representations (std=1.34) ideal for bit extraction. All 33 hidden state outputs are available (embedding + 32 layers).
-### Architecture
-Standard MLP block with parallel circuit path:
-```
-x ──┬── MLP path ────────────────┬── + ── output
-    │                            │
-    └── BitExtractor ── Circuit ─┴── BitInjector
-                          │
-                       Router (learned weighting)
-```
-Augmented MLP forward pass:
-```python
-def forward(x):  # x: [batch, seq, d_model=960]
-    # Original MLP path (unchanged)
-    mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x))
-    # Circuit path (new)
-    a_bits, b_bits = self.bit_extractor(x)       # [batch, seq, 8] each
-    result_bits, carry = self.circuits.add_8bit(a_bits, b_bits)
-    flags = self.compute_flags(result_bits, carry)
-    circuit_delta = self.bit_injector(result_bits, flags)
-    # Routing
-    route_weights = self.router(x)  # [batch, seq, 2] softmax
-    # Combine
-    return mlp_out + route_weights[..., 1:2] * circuit_delta
-```
-### Threshold Logic Fundamentals
-A threshold gate computes:
-```
-output = 1  if  (Σ wᵢxᵢ + b) ≥ 0
-         0  otherwise
-```
-Example gates:
-```
-AND: w=[1,1], b=-2
-  AND(0,0) = H(-2) = 0
-  AND(1,1) = H(0)  = 1
-OR: w=[1,1], b=-1
-  OR(0,1) = H(0) = 1
-  OR(1,1) = H(1) = 1
-XOR: requires 2 layers (not linearly separable)
-  Layer 1: OR + NAND
-  Layer 2: AND
-```
-Full adder = 2 half-adders + carry OR, ~4 threshold layers.
-8-bit ripple carry = 8 chained full adders, ~32 threshold layers.
-### Interface Layers (Trainable)
-**Extractor** — Extracts operands and operation from LLM hidden states:
-```python
-class Extractor(nn.Module):
-    """Attention pooling + per-bit extraction networks."""
-    def __init__(self, hidden_dim=960):
-        self.attention_pool = AttentionPooling(hidden_dim, num_heads=4)
-        self.a_extractor = MultiHeadBitExtractor(hidden_dim)  # 8 separate bit networks
-        self.b_extractor = MultiHeadBitExtractor(hidden_dim)
-        self.op_router = nn.Sequential(
-            nn.Linear(hidden_dim, 256), nn.GELU(),
-            nn.Linear(256, 6)  # 6 operations
-        )
-    def forward(self, hidden_states, attention_mask):
-        pooled = self.attention_pool(hidden_states, attention_mask)  # (batch, 960)
-        a_bits, _ = self.a_extractor(pooled)  # (batch, 8)
-        b_bits, _ = self.b_extractor(pooled)  # (batch, 8)
-        op_logits = self.op_router(pooled)    # (batch, 6)
-        return a_bits, b_bits, op_logits
-```
-**MultiHeadBitExtractor** — 8 specialized networks, one per bit:
-```python
-class MultiHeadBitExtractor(nn.Module):
-    def __init__(self, hidden_dim=960):
-        self.bit_extractors = nn.ModuleList([
-            nn.Sequential(nn.Linear(hidden_dim, 128), nn.GELU(), nn.Linear(128, 1))
-            for _ in range(8)
-        ])
-    def forward(self, x):
-        logits = torch.cat([ext(x) for ext in self.bit_extractors], dim=-1)
-        soft = torch.sigmoid(logits)
-        hard = heaviside_ste(logits)
-        return hard - soft.detach() + soft, logits  # STE
-```
-**AttentionPooling** — Learns which token positions matter:
-```python
-class AttentionPooling(nn.Module):
-    """CLS-token style pooling with learned attention."""
-    def __init__(self, hidden_dim=960, num_heads=4):
-        self.cls_token = nn.Parameter(torch.randn(1, 1, hidden_dim) * 0.02)
-        self.query = nn.Linear(hidden_dim, hidden_dim)
-        self.key = nn.Linear(hidden_dim, hidden_dim)
-        self.value = nn.Linear(hidden_dim, hidden_dim)
-```
-### Trainable Parameters
-For SmolLM2-360M (hidden_dim=960):
-| Component | Parameters | Description |
-|-----------|------------|-------------|
-| AttentionPooling | ~3.7M | 4-head attention over sequence |
-| MultiHeadBitExtractor (×2) | ~245K each | 8 per-bit MLPs for A and B |
-| OpRouter | ~246K | 960→256→6 MLP |
-| **Extractor Total** | ~4.4M | Full extraction module |
-**Alternative architectures**:
-- `PositionExtractor`: ~1.5M (position-specific, no attention)
-- `DigitExtractor`: ~1.2M (predicts digits 0-9 instead of bits)
-With `--unfreeze_layers 4`: Adds ~39.3M trainable params (top 4 transformer layers).
-### Gradient Flow
-Heaviside has zero gradient almost everywhere. We use **Straight-Through Estimator (STE)**:
-```python
-class HeavisideSTE(torch.autograd.Function):
-    @staticmethod
-    def forward(ctx, x):
-        return (x >= 0).float()
-    @staticmethod
-    def backward(ctx, grad_output):
-        return grad_output  # pass through unchanged
-```
-### Training Strategy
-1. **Data**: Random 8-bit arithmetic problems (operands 0-255, 6 operations)
-2. **Loss**: Multi-component BCE + CE
-   - `result_loss`: BCE on output bits vs expected
-   - `a_loss`, `b_loss`: BCE on extracted bits vs ground truth (2× weight)
-   - `op_loss`: CE on operation classification
-3. **Optimizer**: AdamW, lr=3e-4, gradient clipping at 1.0
-4. **Curriculum**: Epoch-based range expansion (0-9 → 0-99 → 0-255)
-5. **Batching**: 256-4096 samples per batch (VRAM-dependent)
-```bash
-# Example training commands
-python train.py --mode router --epochs 100                    # Sanity check
-python train.py --mode llm --epochs 100 --batch_size 256      # Frozen LLM
-python train.py --mode llm --unfreeze_layers 4 --batch_size 4096  # Fine-tune top layers
-```
-### Inference
-At inference, Heaviside is true step function—no approximation. If the Extractor correctly identifies operands, the circuit **will** output the correct result.
-### Target Performance
-| Condition | Configuration | Accuracy |
-|-----------|---------------|----------|
-| Control | Vanilla SmolLM2-360M | 11.90% |
-| Circuits only | Ground truth bits | 100.00% |
-| Experimental | LLM + Extractor + Circuits | **Target: 100%** |
-The interface generalizes to **all** 65,536 8-bit additions once trained—no memorization, the circuits compute.
-### LLM Integration: Proof of Concept (In Progress)
-Before proceeding with architectural extensions, we are validating the core thesis: that frozen threshold circuits can provide exact arithmetic capability to language models that otherwise fail at computation.
-#### Baseline Evaluation
-We evaluated SmolLM2-360M-Instruct on randomized 8-bit arithmetic using a generous answer extraction protocol. The model was prompted with a system message instructing it to output only numeric answers, and we accepted any correct number found in the output (first number, last number, or word-to-number conversion).
-| Operation | SmolLM2-360M Accuracy | Notes |
-|-----------|----------------------|-------|
-| Addition (A + B) | 35.92% | Best performance, still fails 2/3 |
-| Subtraction (A - B) | 17.72% | Poor handling of borrowing |
-| Multiplication (A × B) | **1.25%** | Near-total failure |
-| Greater Than (A > B) | 14.37% | Often echoes expression |
-| Less Than (A < B) | 4.31% | Often echoes expression |
-| Equality (A == B) | 0.28% | Near-total failure |
-| **Overall Fitness** | **11.90%** | 238/2000 correct |
-**Methodology**: 2000 randomized test cases with operands uniformly sampled from [0, 255]. Ground truth computed as 8-bit arithmetic (matching the threshold circuit specification). Batch size 64, greedy decoding (temperature=0).
-**Key Observations**:
-- Multiplication accuracy (1.25%) is essentially random guessing over the output space
-- Comparison operations fail because the model often echoes the expression rather than evaluating it
-- Even addition—the simplest operation—fails nearly two-thirds of the time on 8-bit operands
-- Performance degrades sharply as operand magnitude increases (edge cases like 127+128 are almost never correct)
-These results establish the **control condition** for our experiment.
-#### Experimental Design
-| Condition | Model Configuration | Target Fitness |
-|-----------|---------------------|----------------|
-| **Control** | Vanilla SmolLM2-360M-Instruct | 11.90% (measured) |
-| **Experimental** | SmolLM2-360M + Frozen ThresholdALU + Trained Interface | **100%** |
-The experimental condition adds:
-1. **BitEncoder** (trainable): Projects hidden states → 24 bits (3 × 8-bit operands)
-2. **OpRouter** (trainable): Selects which circuit to activate based on context
-3. **BitDecoder** (trainable): Projects 8-bit result → hidden state delta
-4. **ThresholdALU** (frozen): The verified circuits from this repository
-**Training Signal**: The fitness function itself. We do not provide answer supervision—the model must learn to correctly encode operands and route to circuits such that the frozen circuits produce correct outputs. This is possible because the circuits are proven correct; the interface layers need only learn the encoding/routing protocol.
-**Success Criterion**: If the experimental condition achieves 100% fitness on randomized arithmetic while the control remains at ~12%, this demonstrates:
-1. The frozen threshold circuits provide exact computation
-2. Neural interface layers can learn to use discrete computational substrates
-3. Small language models can achieve perfect arithmetic via architectural augmentation rather than scale
-#### Progress
-**Stage 1: Circuit Validation — COMPLETE**
-The frozen threshold circuits achieve 100% accuracy when given correctly formatted bit inputs:
-| Test | Result |
-|------|--------|
-| DirectCircuitModel (ground truth bits) | 100.00% on 10,000 random cases |
-| All operations (ADD, SUB, MUL, GT, LT, EQ) | 100.00% each |
-This confirms the circuits compute correctly. However, this was already established by `eval.py`.
-**Stage 2: LLM Baseline — COMPLETE**
-SmolLM2-360M-Instruct baseline on randomized 8-bit arithmetic:
-| Operation | Accuracy |
-|-----------|----------|
-| Addition | 35.92% |
-| Subtraction | 17.72% |
-| Multiplication | 1.25% |
-| Comparisons | 0.28–14.37% |
-| **Overall** | **11.90%** |
-Head-to-head on 50 random cases: SmolLM2 got 7/50 (14%), circuits got 50/50 (100%).
-**Stage 3: LLM Integration — IN PROGRESS**
-The challenge: train an interface that extracts operands and operations from natural language (not from pre-formatted bit inputs).
-```
-"47 + 86"
-    ↓
-[Language Model / Extractor]
-    ↓
-[a_bits, b_bits, op_logits]
-    ↓
-[Frozen threshold circuits]
-    ↓
-[Result bits] → 133
-```
-**SmolLM2 Approach** (`llm_integration/`):
-Initial experiments used SmolLM2-360M-Instruct as the language understanding backbone.
-| Mode | Description | Status |
-|------|-------------|--------|
-| `--mode router` | Train OpRouter with ground truth bits | 100% achieved |
-| `--mode interface` | Train BitEncoder + OpRouter | Ready |
-| `--mode llm` | Train from LLM hidden states | Explored |
-**LLM Mode Options**:
-- `--unfreeze_layers N`: Fine-tune top N transformer layers
-- `--extract_layer N`: Extract from intermediate layer (-1 = final)
-- `--position_extract`: Position-specific extraction (uses token positions)
-- `--digit_pred`: Predict digits (0-9) instead of bits
-**Extraction Architectures** (`model.py`):
-- `Extractor`: Attention pooling + per-bit MLPs
-- `PositionExtractor`: Position-aware (operand A from positions 0-2, B from 5-7)
-- `DigitExtractor`: Predicts 3 digits per operand, converts to bits
-- `HybridExtractor`: Digit lookup + MLP fallback for word inputs
-**Curriculum Learning**: Training progresses 0-9 → 0-99 → 0-255 over epochs.
-**Observations**: SmolLM2 integration proved challenging—360M parameters of pre-trained representations largely irrelevant to arithmetic parsing, high VRAM requirements, and gradient conflicts between frozen circuits and pre-trained weights.
-**Pivot: From-Scratch Extractor**
-Given that the task is fundamentally simple—parse `(a, b, op)` from structured text—a lightweight purpose-built model may be more appropriate than adapting a general LLM.
-```
-"one thousand plus two thousand"
-    ↓
-[Char-level tokenizer: ~40 tokens]
-    ↓
-[Small transformer: ~1-5M params]
-    ↓
-[3 heads: a_value, b_value, op_idx]
-    ↓
-[Frozen 32-bit threshold circuits]
-    ↓
-3000
-```
-**Design principles**:
-- **Minimal Python**: All parsing logic learned in weights, not hardcoded
-- **Character-level input**: No word tokenization; model learns "forty seven" = 47
-- **From-scratch training**: No pre-trained weights to conflict with
-- **32-bit target**: Practical arithmetic range (0–4,294,967,295)
-**Planned architecture**:
-- Vocab: ~40 chars (a-z, 0-9, space, operators)
-- Embedding: 40 × 128d
-- Encoder: 2-3 transformer layers
-- Output heads: `a_classifier`, `b_classifier`, `op_classifier`
-- Total: ~1-5M params (vs 360M for SmolLM2)
-This approach treats the problem as what it is: a structured parsing task where the frozen circuits handle all computation. The extractor need only learn the mapping from text to operands—no world knowledge required.
-#### Proof of Concept Scope
-- **32-bit operands** (0–4,294,967,295)
-- **Six operations**: ADD, SUB, MUL, GT, LT, EQ
-- **Structured input**: Digits ("1000 + 2000") and number words ("one thousand plus two thousand")
-**Current Status**:
-- Circuit validation: Complete (100% on 8-bit operations)
-- 32-bit circuits: Built and tested (adder verified on 1M+2M=3M, etc.)
-- LLM baseline: Measured (11.90% - establishes control condition)
-- SmolLM2 integration: Infrastructure complete, training explored
-- From-scratch extractor: Design phase
-### Extension Roadmap
-#### Completed
-1. **32-bit operations (0–4,294,967,295)** — Full 32-bit ALU implemented via `--bits 32` flag:
-   - 32-bit ripple carry adder (32 chained full adders) — **verified**
-   - 32-bit subtractor (NOT + adder with carry-in)
-   - 32-bit multiplication (1024 partial product ANDs)
-   - 32-bit division (32 restoring stages)
-   - 32-bit comparators (GT, LT, GE, LE, EQ)
-   - 32-bit bitwise ops (AND, OR, XOR, NOT)
-   - 32-bit shifts (SHL, SHR), INC, DEC, NEG
-   **Known issue**: Single-layer 32-bit comparators use weights up to 2³¹, which exceeds float32 mantissa precision (24 bits). Comparisons between large numbers differing only in low bits may fail. Fix planned: cascaded byte-wise comparison (compare MSB first, if equal compare next byte, etc.).
-2. **3-operand addition (15 + 27 + 33 = 75)** — `arithmetic.add3_8bit` chains two 8-bit ripple carry stages. 16 full adders, 144 gates, 240 test cases verified.
-3. **Order of operations (5 + 3 × 2 = 11)** — `arithmetic.expr_add_mul` computes A + (B × C) using shift-add multiplication then addition. 64 AND gates + 64 full adders, 73 test cases verified.
-#### Planned
-1. **Cascaded 32-bit comparators** — Replace single-layer weighted comparison with multi-layer byte-wise cascade. Each byte comparison uses 8-bit weights (max 128), well within float32 precision. Hardware-accurate and extensible to 64-bit, 128-bit, etc.
-2. **Parenthetical expressions ((5 + 3) × 2 = 16)** — Explicit grouping overrides precedence. Parser must recognize parens and build correct tree. Evaluation proceeds innermost-out.
-3. **Multi-operation chains (a + b - c × d)** — Sequential dispatch through multiple circuits with intermediate result routing. Requires state management in interface layers.
-4. **Floating point arithmetic** — IEEE 754-style with separate circuits for mantissa and exponent. ADD: align exponents, add mantissas, renormalize. MUL: add exponents, multiply mantissas.
-5. **Full CPU integration** — Enable memory access circuits for stateful computation. Allows multi-step algorithms executed entirely within threshold logic.
----
-## Build Tool
-Output filenames are auto-generated from configuration:
-```
-Format: neural_{alu|computer}{BITS}[_{MEMORY}].safetensors
-Examples:
-  neural_alu8.safetensors               # 8-bit, no memory
-  neural_alu32.safetensors              # 32-bit, no memory
-  neural_computer8.safetensors          # 8-bit, full memory (default)
-  neural_computer32.safetensors         # 32-bit, full memory
-  neural_computer8_small.safetensors    # 8-bit, 1KB memory
-  neural_computer32_small.safetensors   # 32-bit, 1KB memory
-  neural_computer8_addr12.safetensors   # 8-bit, custom 4KB (2^12 bytes)
-```
-```bash
-# 8-bit CPU (default)
-python build.py --apply all                                # -> neural_computer8.safetensors
-python build.py -m none --apply all                        # -> neural_alu8.safetensors
-python build.py -m scratchpad --apply all                  # -> neural_computer8_scratchpad.safetensors
-# 16-bit ALU
-python build.py --bits 16 --apply all                      # -> neural_computer16.safetensors
-python build.py --bits 16 -m none --apply all              # -> neural_alu16.safetensors
-# 32-bit ALU
-python build.py --bits 32 -m small --apply all             # -> neural_computer32_small.safetensors
-python build.py --bits 32 -m none --apply all              # -> neural_alu32.safetensors
-# Custom address width
-python build.py --bits 16 -a 6 --apply all                 # -> neural_computer16_addr6.safetensors
-```
-**Bit widths** (`--bits`):
-| Width | Range | Use Case |
-|-------|-------|----------|
-| 8 | 0–255 | Full CPU, legacy |
-| 16 | 0–65,535 | Extended arithmetic |
-| 32 | 0–4,294,967,295 | Practical arithmetic |
-**Memory profiles** (`-m`):
-| Profile | Size | Addr Bits | Filename Suffix | Params | Use Case |
-|---------|------|-----------|-----------------|--------|----------|
-| `none` | 0B | — | (uses `alu`) | ~32K | Pure ALU |
-| `registers` | 16B | 4 | `_registers` | ~34K | Minimal state |
-| `scratchpad` | 256B | 8 | `_scratchpad` | ~63K | 8-bit scratch |
-| `small` | 1KB | 10 | `_small` | ~123K | 32-bit scratch |
-| `reduced` | 4KB | 12 | `_reduced` | ~549K | Small programs |
-| `full` | 64KB | 16 | (none) | ~8.29M | Full CPU |
-**Custom address width** (`-a N`): Memory size = 2^N bytes, suffix = `_addrN`
----
-## Citation
-```bibtex
-@misc{8bit-threshold-computer,
-  title={8bit-threshold-computer: A Turing-Complete Threshold Logic CPU},
-  author={Norton, Charles},
-  year={2026},
-  howpublished={Hugging Face},
-  url={https://huggingface.co/phanerozoic/8bit-threshold-computer}
-}
-```
----
-## License
-MIT
----
-## References
-1. McCulloch & Pitts (1943). "A Logical Calculus of Ideas Immanent in Nervous Activity"
-2. Muroga (1971). "Threshold Logic and Its Applications"
-3. Siegelmann & Sontag (1995). "On the Computational Power of Neural Nets"
-4. Bengio et al. (2013). "Estimating or Propagating Gradients Through Stochastic Neurons"
-5. Ma et al. (2024). "The Era of 1-bit LLMs" (BitNet b1.58)
-6. HuggingFace (2024). "SmolLM2: Small Language Models" — [Model Card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct)
-7. Vaswani et al. (2017). "Attention Is All You Need" — Transformer architecture
-8. Su et al. (2021). "RoFormer: Enhanced Transformer with Rotary Position Embedding" — RoPE

+---
+license: mit
+tags:
+- threshold-logic
+- neuromorphic
+- computer-architecture
+- turing-complete
+- loihi
+- truenorth
+- akida
+---
+# 8bit-threshold-computer
+A Turing-complete CPU implemented entirely as threshold logic gates. Every gate, from Boolean primitives to arithmetic to control flow, is a single threshold neuron of the form:
+```
+output = 1 if (Σ wᵢ·xᵢ + b) ≥ 0 else 0
+```
+Weights and biases are integers; activations are the Heaviside step. Nothing else.
+The repository ships eighteen prebuilt configurations spanning three data-path widths (8, 16, 32 bits) and six memory sizes (0 B to 64 KB). The canonical file at the repo root is the largest of these: a 32-bit data path with a 64 KB address space and ~8.47 M parameters.
+```
+neural_computer.safetensors        32-bit data, 64 KB memory, ~8.47M params (canonical)
+variants/neural_computer{8,16,32}.safetensors                    full memory (64 KB)
+variants/neural_computer{8,16,32}_reduced.safetensors            4 KB memory
+variants/neural_computer{8,16,32}_small.safetensors              1 KB memory
+variants/neural_computer{8,16,32}_scratchpad.safetensors         256 B memory
+variants/neural_computer{8,16,32}_registers.safetensors          16 B memory
+variants/neural_alu{8,16,32}.safetensors                         pure ALU, no memory
+```
+---
+## Quick start
+```python
+import torch
+from safetensors.torch import load_file
+tensors = load_file("neural_computer.safetensors")
+def heaviside(x):
+    return (x >= 0).float()
+# AND gate: fires when both inputs are 1
+w = tensors['boolean.and.weight']  # [2]
+b = tensors['boolean.and.bias']    # [1]
+for a, c in [(0, 0), (0, 1), (1, 0), (1, 1)]:
+    out = heaviside((torch.tensor([a, c], dtype=torch.float32) * w).sum() + b)
+    print(f"AND({a}, {c}) = {int(out.item())}")
+```
+Run the full circuit verification suite against any variant:
+```bash
+python eval_all.py variants/                              # all 18 in one pass
+python eval_all.py neural_computer.safetensors            # the canonical file
+python eval_all.py --cpu-program variants/                # also run an assembled
+                                                          # program through the
+                                                          # threshold-gated CPU
+```
+`eval_all.py` reads each variant's manifest, runs a gate-level fitness suite (5,900–7,800 tests per variant covering Boolean, arithmetic, ALU, control, modular, error-detection, threshold, and IEEE 754 float circuits), and optionally executes a small assembled program through a manifest-sized threshold CPU plus a chained 16- or 32-bit ALU sequence on wider variants.
+For an interactive walkthrough that exercises Boolean gates, the 8-bit ALU, mod-5 divisibility, and a CPU loop end-to-end:
+```bash
+python play.py                                            # 1 KB demo, runs in seconds
+python play.py --model neural_computer.safetensors        # 64 KB, slower
+```
+---
+## Execution model
+A self-contained machine. State goes in, state comes out:
+- **Pure tensor computation**: state in, state out
+- **Frozen circuits**: integer weights, Heaviside activation
+- **ACT execution**: internal loop until `HALT`
+- **No external orchestration**: one forward pass equals one complete program execution
+```
+            ┌─────────────────────────────┐
+            │      Initial State          │
+            │  [PC|Regs|Flags|Memory...]  │
+            └─────────────┬───────────────┘
+                          ▼
+            ┌─────────────────────────────┐
+            │   Threshold Circuit Layer   │
+            │  ┌───────────────────────┐  │
+            │  │   Fetch: PC → Instr   │  │
+            │  ├───────────────────────┤  │
+            │  │   Decode: Opcode/Ops  │  │
+            │  ├───────────────────────┤  │
+            │  │   Execute: ALU/Mem    │  │
+            │  ├───────────────────────┤  │
+            │  │   Writeback: Results  │  │
+            │  ├───────────────────────┤  │
+            │  │   PC Update           │  │
+            │  └───────────┬───────────┘  │
+            │              │              │
+            │         ┌────▼────┐         │
+            │         │ HALTED? │         │
+            │         └────┬────┘         │
+            │       no ────┴──── yes      │
+            │        │           │        │
+            │        ▼           ▼        │
+            │     [loop]      [exit]      │
+            └─────────────┬───────────────┘
+                          ▼
+            ┌─────────────────────────────┐
+            │       Final State           │
+            └─────────────────────────────┘
+```
+### Instruction set
+| Opcode | Mnemonic | Operation |
+|--------|----------|-----------|
+| 0x0 | ADD | R[d] = R[a] + R[b] |
+| 0x1 | SUB | R[d] = R[a] - R[b] |
+| 0x2 | AND | R[d] = R[a] & R[b] |
+| 0x3 | OR | R[d] = R[a] \| R[b] |
+| 0x4 | XOR | R[d] = R[a] ^ R[b] |
+| 0x5 | SHL | R[d] = R[a] << 1 |
+| 0x6 | SHR | R[d] = R[a] >> 1 |
+| 0x7 | MUL | R[d] = R[a] * R[b] |
+| 0x8 | DIV | R[d] = R[a] / R[b] |
+| 0x9 | CMP | flags = R[a] - R[b] |
+| 0xA | LOAD | R[d] = M[addr] |
+| 0xB | STORE | M[addr] = R[s] |
+| 0xC | JMP | PC = addr |
+| 0xD | Jcc | PC = addr if cond (imm8[2:0]: 0=Z, 1=NZ, 2=C, 3=NC, 4=N, 5=P, 6=V, 7=NV) |
+| 0xE | CALL | push PC; PC = addr |
+| 0xF | HALT | stop execution |
+### State tensor layout
+All multi-bit fields are MSB-first (index 0 is the most-significant bit).
+```
+[ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ]
+```
+`N` is the address width (configurable, 0–16). Flags are ordered `Z, N, C, V`. Control bits are ordered `HALT, MEM_WE, MEM_RE, RESERVED`.
+### Instruction encoding (16-bit, MSB-first)
+```
+15..12  11..10  9..8  7..0
+opcode  rd      rs    imm8
+```
+Interpretation:
+- **R-type**: `rd = rd op rs` (imm8 ignored)
+- **I-type**: `rd = op rd, imm8` (rs ignored)
+- **Address-extended**: `LOAD`, `STORE`, `JMP`, `Jcc`, `CALL` consume the next word as a 16-bit address (big-endian); `imm8` is reserved and the PC skips 4 bytes when the jump is not taken.
+### Circuit categories
+| Category | Circuits | Examples |
+|----------|----------|----------|
+| Boolean | 9 | AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES |
+| Arithmetic | 18+ | half/full adder, ripple-carry (8/16/32-bit), comparators (8/16/32-bit), 3-operand adder, A+B×C and (A+B)×C expressions |
+| ALU | 8/16/32-bit | shifts, multiply, divide, INC/DEC, NEG, ROL/ROR, bitwise |
+| Combinational | 10+ | MUX (2:1, 4:1, 8:1), DEMUX, 3-to-8 decoder, 8-to-3 encoder, barrel shifter, priority encoder |
+| Control flow | 16 | JMP, conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV), CALL, RET, PUSH, POP |
+| Memory | 3 | N-bit address decoder, read mux, write cells (packed) |
+| Modular | 11 | divisibility by 2–12 (multi-layer for non-powers-of-2) |
+| Threshold | 13 | k-of-n gates, majority, minority, exactly-k |
+| Pattern | 10 | popcount, leading/trailing ones, symmetry |
+| Error detection | 11 | parity (XOR tree), checksum, CRC, Hamming |
+| Float (IEEE 754) | half + single | pack/unpack, classify, normalize, ADD, MUL, DIV, EQ/LT/LE/GT/GE |
+### Tensor naming
+```
+{category}.{circuit}[.{layer}][.{component}].{weight|bias}
+Examples:
+  boolean.and.weight
+  boolean.xor.layer1.neuron1.weight
+  arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight
+  modular.mod5.layer2.eq3.weight
+  error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias
+```
+Memory circuits are stored as packed tensors so the safetensors header stays manageable (`memory.addr_decode.weight`, `memory.read.and.weight`, `memory.write.and_old.weight`, etc.).
+---
+## Bit widths and memory profiles
+The build tool emits one of 51 functionally distinct configurations: three data-path widths × seventeen address widths (0–16, where 0 means no memory).
+**Bit widths** (`--bits`):
+| Width | Range | Use case |
+|-------|-------|----------|
+| 8 | 0–255 | full CPU, legacy compatibility |
+| 16 | 0–65,535 | extended arithmetic |
+| 32 | 0–4,294,967,295 | practical arithmetic ranges |
+**Memory profiles** (`-m`):
+| Profile | Size | Addr bits | Filename suffix |
+|---------|------|-----------|-----------------|
+| `none` | 0 B | 0 | (uses `alu` instead of `computer`) |
+| `registers` | 16 B | 4 | `_registers` |
+| `scratchpad` | 256 B | 8 | `_scratchpad` |
+| `small` | 1 KB | 10 | `_small` |
+| `reduced` | 4 KB | 12 | `_reduced` |
+| `full` | 64 KB | 16 | (none) |
+Auto-generated filename: `neural_{alu|computer}{BITS}[_{MEMORY}].safetensors`. Custom address widths via `-a N` produce `_addrN`.
+```bash
+python build.py --bits 32 --apply all              # neural_computer32.safetensors
+python build.py --bits 8 -m none --apply all       # neural_alu8.safetensors
+python build.py --bits 16 -m small --apply all     # neural_computer16_small.safetensors
+python build.py --bits 32 -a 6 --apply all         # neural_computer32_addr6.safetensors
+```
+To regenerate every named variant in one pass:
+```bash
+python build_all.py
+```
+This populates `variants/` with all 18 builds and runs `eval.py` on each as a sanity check.
+---
+## Verification
+| Category | Status | Notes |
+|----------|--------|-------|
+| Boolean gates | exhaustive | all 2^n input combinations |
+| Arithmetic | exhaustive | full 8-bit range; strategic sampling at 16/32-bit |
+| ALU | exhaustive | every operation, every input |
+| Control flow | exhaustive | branch and jump conditions |
+| Threshold | exhaustive | k-of-n, majority, etc. |
+| Modular (mod 3, 5, 6, 7, 9, 10, 11, 12) | exhaustive | multi-layer, hand-constructed |
+| Parity | exhaustive | XOR tree, hand-constructed |
+| Modular (mod 2, 4, 8) | exhaustive | single-layer, trivial |
+Divisibility by non-powers-of-2 (3, 5, 7, ...) is not linearly separable in binary, so those circuits are multi-layer. Eight-bit parity (XOR of all bits) requires a tree of XOR gates. All circuits pass exhaustive testing over their full input domains.
+`eval_all.py` runs the unified suite. Exit code is the number of failing variants (0 means all pass).
+---
+## Threshold logic
+A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing the result to a threshold; the output is 1 when the sum meets or exceeds the threshold and 0 otherwise. Equivalently, it is a neuron with Heaviside step activation, integer weights, and an integer bias.
+Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function, which includes AND, OR, NAND, NOR, IMPLIES, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (XOR, parity, mod-k for k not a power of two) require multiple layers.
+Example gates:
+```
+AND: w=[1, 1], b=-2
+  H(0+0-2) = 0     H(1+1-2) = 1
+OR:  w=[1, 1], b=-1
+  H(0+0-1) = 0     H(1+0-1) = 1
+XOR: two layers (not linearly separable)
+  layer 1: OR + NAND
+  layer 2: AND of the two
+```
+A full adder is two half-adders plus a carry OR, around four threshold layers. An 8-bit ripple-carry adder is eight chained full adders, around 32 layers.
+### History
+Warren McCulloch and Walter Pitts introduced the threshold neuron in 1943, proving that networks of such neurons can compute any Boolean function. Their work preceded both the perceptron and modern neural networks and established the theoretical foundation for neural computation.
+The 1960s saw substantial work on threshold logic synthesis. Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function can be implemented as a single threshold gate, and if so, how to compute the appropriate weights. The focus was on individual gates rather than complete systems.
+Frank Rosenblatt's Mark I Perceptron (1957–1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training; the final configurations were not published. Bernard Widrow's ADALINE and MADALINE (1960–1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm.
+Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing-complete. The construction relied on continuous sigmoid activations with infinite precision, not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: universality with continuous activations chosen to support gradient-based training.
+### Neuromorphic hardware
+Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon:
+- **Intel Loihi** (2017): 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. Supports integer weights and configurable neuron dynamics.
+- **IBM TrueNorth** (2014): one million neurons and 256 million synapses across a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds.
+- **BrainChip Akida** (2021): edge-oriented event-based processing with integer weights.
+- **SpiNNaker** (University of Manchester): ARM cores simulating spiking networks at scale.
+Published work on these platforms has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1 and described that work as "a first step toward the construction of a spiking computer"; that implementation lacked instruction fetch, a program counter, memory, and control logic.
+The weights in this repository implement a complete CPU: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every component is a threshold neuron with integer weights.
+---
+## Hardware compatibility
+All weights are integers, all activations are Heaviside step, and every gate is a single weighted sum. The circuits are intended to deploy directly on:
+- **Intel Loihi**
+- **IBM TrueNorth**
+- **BrainChip Akida**
+---
+## LLM integration
+Threshold circuits can be embedded into transformer MLP layers to give a language model exact arithmetic. Standard LLMs fail at arithmetic because they interpolate over the training distribution rather than compute, so a 360M-parameter model trained on web text has seen `127 + 128 = 255` few times if at all and guesses based on pattern matching.
+The integration freezes the circuits and trains only the interface layers that:
+1. Extract operands from token embeddings.
+2. Route computation through the appropriate circuit.
+3. Inject the result back into the residual stream.
+The model learns *call dispatch*; the arithmetic is already solved.
+### Architecture
+```
+x ──┬── MLP path ─────────────────┬── + ── output
+    │                             │
+    └── BitExtractor ── Circuit ──┴── BitInjector
+                          │
+                       Router (learned weighting)
+```
+Augmented MLP forward pass:
+```python
+def forward(x):  # x: [batch, seq, d_model=960]
+    mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x))
+    a_bits, b_bits = self.bit_extractor(x)              # [batch, seq, 8] each
+    result_bits, carry = self.circuits.add_8bit(a_bits, b_bits)
+    flags = self.compute_flags(result_bits, carry)
+    circuit_delta = self.bit_injector(result_bits, flags)
+    route_weights = self.router(x)                       # [batch, seq, 2] softmax
+    return mlp_out + route_weights[..., 1:2] * circuit_delta
+```
+### Target model
+The reference integration uses HuggingFace's [SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct). See [`llm_integration/SMOLLM2_ARCHITECTURE.md`](llm_integration/smollm2/SMOLLM2_ARCHITECTURE.md) for the full technical analysis.
+| Property | Value |
+|----------|-------|
+| Parameters | 361.82 M |
+| Hidden dimension | 960 (matches the extractor input) |
+| Layers | 32 transformer blocks |
+| Attention | 15 query heads, 5 KV heads (GQA) |
+| MLP | SwiGLU (960 → 2560 → 960) |
+| Position encoding | RoPE (theta = 100k, max 8192) |
+Digits tokenize individually (`"47 + 86"` → `['4', '7', ' +', ' ', '8', '6']`, with digit token IDs `32 + digit_value`), which makes position-based operand extraction practical.
+### Gradient flow
+Heaviside has zero gradient almost everywhere. The implementation uses a straight-through estimator:
+```python
+class HeavisideSTE(torch.autograd.Function):
+    @staticmethod
+    def forward(ctx, x):
+        return (x >= 0).float()
+    @staticmethod
+    def backward(ctx, grad_output):
+        return grad_output
+```
+At inference, Heaviside is the true step function; if the extractor identifies operands correctly, the circuit produces the correct result by construction.
+### Baseline
+SmolLM2-360M-Instruct on randomized 8-bit arithmetic (2,000 cases, operands uniform on [0, 255], generous answer extraction):
+| Operation | Accuracy |
+|-----------|----------|
+| Addition | 35.92% |
+| Subtraction | 17.72% |
+| Multiplication | 1.25% |
+| Greater than | 14.37% |
+| Less than | 4.31% |
+| Equality | 0.28% |
+| **Overall** | **11.90%** (238/2000) |
+Multiplication accuracy at 1.25% is essentially random over the output space. Comparison operations often echo the expression rather than evaluate it. Even addition fails roughly two-thirds of the time on full 8-bit operands. Performance degrades further as operand magnitude increases: edge cases like `127 + 128` are almost never correct.
+The frozen threshold circuits reach 100% on the same task when given correctly formatted bit inputs (10,000 random cases, every operation). The integration challenge is therefore the extractor, not the arithmetic.
+### Trainable parameters (SmolLM2, hidden_dim = 960)
+| Component | Parameters | Description |
+|-----------|------------|-------------|
+| AttentionPooling | ~3.7 M | 4-head attention over the sequence |
+| MultiHeadBitExtractor (× 2) | ~245 K each | 8 per-bit MLPs for A and B |
+| OpRouter | ~246 K | 960 → 256 → 6 MLP |
+| **Extractor total** | **~4.4 M** | full extraction module |
+Alternative architectures: `PositionExtractor` (~1.5 M, position-specific, no attention), `DigitExtractor` (~1.2 M, predicts digits 0–9 instead of bits), `HybridExtractor` (digit lookup with MLP fallback for word numerals). With `--unfreeze_layers 4` an additional ~39.3 M trainable parameters open up in the top four transformer layers.
+### Training
+```bash
+python train.py --mode router --epochs 100                          # sanity check
+python train.py --mode llm --epochs 100 --batch_size 256            # frozen LLM
+python train.py --mode llm --unfreeze_layers 4 --batch_size 4096    # fine-tune top layers
+```
+Loss components: BCE on output bits, BCE on extracted A and B bits (2× weight), and CE on operation classification. Curriculum runs 0–9 → 0–99 → 0–255. Optimizer is AdamW, lr 3e-4, gradient clipping 1.0.
+---
+## Repository layout
+```
+neural_computer.safetensors         canonical model (32-bit, 64 KB, ~8.47M params)
+variants/                           18 prebuilt configurations
+build.py                            generator (one safetensors per invocation)
+build_all.py                        builds and verifies every named profile
+eval.py                             gate-level fitness suite + reference CPU runtime
+eval_all.py                         variant-agnostic harness (recommended)
+play.py                             interactive demo
+prune_weights.py                    GPU-batched weight reduction with conflict resolution
+llm_integration/                    SmolLM2 extractor + circuit wrapper + training code
+  ├── circuits.py                   FrozenThresholdCircuits (loads safetensors, exposes
+  │                                 add_8bit / sub_8bit / mul_8bit / compare_*)
+  ├── model.py                      Extractor variants, ArithmeticModel
+  ├── train.py                      router / interface / llm training modes
+  ├── fitness.py                    randomized fitness function
+  ├── baseline.py                   vanilla SmolLM2 baseline measurement
+  ├── trained/                      checkpointed extractor weights
+  └── smollm2/
+      ├── SMOLLM2_ARCHITECTURE.md   architecture analysis
+      ├── analyze_smollm2.py        analysis script
+      └── smollm2_analysis.json     analysis output
+```
+---
+## Citation
+```bibtex
+@misc{8bit-threshold-computer,
+  title={8bit-threshold-computer: A Turing-Complete Threshold Logic CPU},
+  author={Norton, Charles},
+  year={2026},
+  howpublished={Hugging Face},
+  url={https://huggingface.co/phanerozoic/8bit-threshold-computer}
+}
+```
+---
+## License
+MIT
+---
+## References
+1. McCulloch & Pitts (1943). *A Logical Calculus of Ideas Immanent in Nervous Activity.*
+2. Muroga (1971). *Threshold Logic and Its Applications.*
+3. Siegelmann & Sontag (1995). *On the Computational Power of Neural Nets.*
+4. Bengio et al. (2013). *Estimating or Propagating Gradients Through Stochastic Neurons.*
+5. Ma et al. (2024). *The Era of 1-bit LLMs* (BitNet b1.58).
+6. HuggingFace (2024). *SmolLM2: Small Language Models* — [model card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct).
+7. Vaswani et al. (2017). *Attention Is All You Need.*
+8. Su et al. (2021). *RoFormer: Enhanced Transformer with Rotary Position Embedding.*

neural_alu32.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2277b9b7ca05aeca4b84da8f8cf48c8ceba9d81ea926a2a1f6be46462fdc9944
-size 10082208

neural_computer.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a76635005d7031f01492b1c5d6286dbede39c1ecf08ed4b08daf4e7c3c2fe097
-size 8435820

 version https://git-lfs.github.com/spec/v1
+oid sha256:18f4f3420fb307d90ea7a8fe356c196a59d7a0f2ed4ec57679d87b209a7fec22
+size 47693920

play.py CHANGED Viewed

@@ -1,25 +1,34 @@
 """
 Hands-on playground for the 8bit-threshold-computer.
-Loads the bundled safetensors model, reads its manifest, and exercises
-threshold circuits at every level: raw Boolean gates, ALU arithmetic,
-comparators, then a CPU runtime sized to the actual manifest that runs
-a small assembled program end-to-end through the threshold weights.
 """
 from __future__ import annotations
 import os
 import sys
 import torch
 from safetensors import safe_open
-sys.path.insert(0, os.path.dirname(__file__))
-# ---------------------------------------------------------------------------
-# Load model + manifest
-# ---------------------------------------------------------------------------
-MODEL_PATH = os.path.join(os.path.dirname(__file__), "neural_computer.safetensors")
 def heaviside(x):
@@ -34,451 +43,202 @@ def load_tensors(path):
     return out
-print("Loading", MODEL_PATH)
-T = load_tensors(MODEL_PATH)
-DATA_BITS = int(T["manifest.data_bits"].item())
-ADDR_BITS = int(T["manifest.addr_bits"].item())
-MEM_BYTES = int(T["manifest.memory_bytes"].item())
-REGISTERS = int(T["manifest.registers"].item())
-print(f"Manifest: data={DATA_BITS}-bit, addr={ADDR_BITS}-bit, mem={MEM_BYTES}B, regs={REGISTERS}")
-print(f"Tensors: {len(T):,}")
-print(f"Total params: {sum(t.numel() for t in T.values()):,}")
-print()
-def gate(name, inputs):
-    """Run one threshold gate identified by `name` (no .weight/.bias suffix)."""
-    w = T[name + ".weight"].view(-1)
-    b = T[name + ".bias"].view(-1)
-    inp = torch.tensor(inputs, dtype=torch.float32)
-    return int(heaviside((inp * w).sum() + b).item())
-def xor(prefix, inputs):
-    """Run a 2-layer XOR-style gate (or/nand naming, e.g. ripple-carry adders)."""
-    a, b_ = inputs
-    h_or = gate(f"{prefix}.layer1.or", [a, b_])
-    h_nand = gate(f"{prefix}.layer1.nand", [a, b_])
-    return gate(f"{prefix}.layer2", [h_or, h_nand])
-def xor_neuron(prefix, inputs):
-    """Run a 2-layer XOR-style gate (neuron1/neuron2 naming, e.g. boolean.xor)."""
-    a, b_ = inputs
-    h1 = gate(f"{prefix}.layer1.neuron1", [a, b_])
-    h2 = gate(f"{prefix}.layer1.neuron2", [a, b_])
-    return gate(f"{prefix}.layer2", [h1, h2])
-def int_to_bits_msb(v, n):
-    return [(v >> (n - 1 - i)) & 1 for i in range(n)]
-def bits_msb_to_int(bits):
-    out = 0
-    for b in bits:
-        out = (out << 1) | int(b)
-    return out
-# ---------------------------------------------------------------------------
-# Demo 1: Boolean gates (README Usage example)
-# ---------------------------------------------------------------------------
-print("=" * 64)
-print(" Demo 1: Boolean threshold gates")
-print("=" * 64)
-truth_2 = [(0, 0), (0, 1), (1, 0), (1, 1)]
-for gname in ["and", "or", "nand", "nor", "implies"]:
-    row = " ".join(f"{a}{b}->{gate(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
-    print(f"  {gname:8} {row}")
-# 2-layer gates (boolean.* uses neuron1/neuron2 naming)
-for gname in ["xor", "xnor", "biimplies"]:
-    row = " ".join(f"{a}{b}->{xor_neuron(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
-    print(f"  {gname:8} {row}")
-# NOT (1-input)
-print(f"  not      0->{gate('boolean.not', [0])} 1->{gate('boolean.not', [1])}")
-print()
-# ---------------------------------------------------------------------------
-# Demo 2: 8-bit ALU operations via threshold weights
-# ---------------------------------------------------------------------------
-print("=" * 64)
-print(" Demo 2: 8-bit ALU arithmetic (every gate is threshold logic)")
-print("=" * 64)
-def fa(prefix, a, b, cin):
-    s1 = xor(f"{prefix}.ha1.sum", [a, b])
-    c1 = gate(f"{prefix}.ha1.carry", [a, b])
-    s2 = xor(f"{prefix}.ha2.sum", [s1, cin])
-    c2 = gate(f"{prefix}.ha2.carry", [s1, cin])
-    cout = gate(f"{prefix}.carry_or", [c1, c2])
-    return s2, cout
-def alu_add(a, b):
-    """8-bit ripple carry add via threshold full-adders."""
-    a_bits = int_to_bits_msb(a, 8)
-    b_bits = int_to_bits_msb(b, 8)
-    a_lsb_first = list(reversed(a_bits))
-    b_lsb_first = list(reversed(b_bits))
-    carry = 0
-    sum_lsb_first = []
-    for i in range(8):
-        s, carry = fa(f"arithmetic.ripplecarry8bit.fa{i}", a_lsb_first[i], b_lsb_first[i], carry)
-        sum_lsb_first.append(s)
-    return bits_msb_to_int(list(reversed(sum_lsb_first))), carry
-def alu_sub(a, b):
-    """A - B via two's complement; uses sub8bit circuit family."""
-    a_lsb = list(reversed(int_to_bits_msb(a, 8)))
-    b_lsb = list(reversed(int_to_bits_msb(b, 8)))
-    carry = 1
-    diff_lsb = []
-    for i in range(8):
-        notb = gate(f"arithmetic.sub8bit.notb{i}", [b_lsb[i]])
-        x1 = xor(f"arithmetic.sub8bit.fa{i}.xor1", [a_lsb[i], notb])
-        x2 = xor(f"arithmetic.sub8bit.fa{i}.xor2", [x1, carry])
-        and1 = gate(f"arithmetic.sub8bit.fa{i}.and1", [a_lsb[i], notb])
-        and2 = gate(f"arithmetic.sub8bit.fa{i}.and2", [x1, carry])
-        carry = gate(f"arithmetic.sub8bit.fa{i}.or_carry", [and1, and2])
-        diff_lsb.append(x2)
-    return bits_msb_to_int(list(reversed(diff_lsb))), carry
-def alu_compare(a, b, kind):
-    """8-bit comparators (single-layer GT/LT, two-layer EQ)."""
-    a_bits = int_to_bits_msb(a, 8)
-    b_bits = int_to_bits_msb(b, 8)
-    inp = a_bits + b_bits
-    if kind == "eq":
-        h_geq = gate("arithmetic.equality8bit.layer1.geq", inp)
-        h_leq = gate("arithmetic.equality8bit.layer1.leq", inp)
-        return gate("arithmetic.equality8bit.layer2", [h_geq, h_leq])
-    return gate(f"arithmetic.{kind}8bit", inp)
-def alu_mul(a, b):
-    """Shift-add multiply via partial-product threshold AND gates + repeated add."""
-    a_bits = int_to_bits_msb(a, 8)
-    b_bits = int_to_bits_msb(b, 8)
-    pp = [[0] * 8 for _ in range(8)]
-    for i in range(8):
-        for j in range(8):
-            pp[i][j] = gate(f"alu.alu8bit.mul.pp.a{i}b{j}", [a_bits[i], b_bits[j]])
-    # accumulate weighted partial products in 8 bits (drop overflow above bit 7)
-    result = 0
-    for j in range(8):  # j=0 is MSB of b -> weight 7-j
-        if b_bits[j] == 0:
-            continue
-        row = 0
-        for i in range(8):
-            row |= (pp[i][j] << (7 - i))
-        shift = 7 - j
-        result, _ = alu_add(result & 0xFF, (row << shift) & 0xFF)
-    return result & 0xFF
-cases_arith = [(5, 3), (37, 100), (200, 99), (255, 1), (127, 128), (15, 17)]
-print("ADD:")
-for a, b in cases_arith:
-    r, c = alu_add(a, b)
-    expect = (a + b) & 0xFF
-    ok = "OK" if r == expect else "FAIL"
-    print(f"  {a:3} + {b:3} = {r:3} (carry={c})  expected {expect:3}  [{ok}]")
-print("SUB:")
-for a, b in cases_arith:
-    r, c = alu_sub(a, b)
-    expect = (a - b) & 0xFF
-    ok = "OK" if r == expect else "FAIL"
-    print(f"  {a:3} - {b:3} = {r:3} (no_borrow={c})  expected {expect:3}  [{ok}]")
-print("CMP:")
-cmp_cases = [(50, 30), (30, 50), (77, 77), (255, 0), (0, 255), (128, 127)]
-for a, b in cmp_cases:
-    gt = alu_compare(a, b, "greaterthan")
-    lt = alu_compare(a, b, "lessthan")
-    eq = alu_compare(a, b, "eq")
-    print(f"  {a:3} vs {b:3} -> GT={gt} LT={lt} EQ={eq}")
-print("MUL (low 8 bits):")
-for a, b in [(12, 11), (15, 17), (8, 32), (200, 3), (0, 99), (1, 255)]:
-    r = alu_mul(a, b)
-    expect = (a * b) & 0xFF
-    ok = "OK" if r == expect else "FAIL"
-    print(f"  {a:3} * {b:3} = {r:3}  expected {expect:3}  [{ok}]")
-print()
-# ---------------------------------------------------------------------------
-# Demo 3: A 4-bit divisibility test (mod 5) - non-linearly-separable
-# ---------------------------------------------------------------------------
-print("=" * 64)
-print(" Demo 3: mod-5 divisibility (multi-layer, hand-constructed)")
-print("=" * 64)
-# layer1: per-residue geq/leq -> layer2: eq -> layer3: OR all eq's
-def mod5(v):
-    bits = int_to_bits_msb(v, 8)
-    # discover number of geq/leq neurons
-    n = 0
-    while f"modular.mod5.layer1.geq{n}.weight" in T:
-        n += 1
-    eqs = []
-    for i in range(n):
-        h_geq = gate(f"modular.mod5.layer1.geq{i}", bits)
-        h_leq = gate(f"modular.mod5.layer1.leq{i}", bits)
-        eqs.append(gate(f"modular.mod5.layer2.eq{i}", [h_geq, h_leq]))
-    return gate("modular.mod5.layer3.or", eqs)
-hits = [v for v in range(256) if mod5(v)]
-print(f"  v in [0,255] with mod5(v)==1: {len(hits)} hits, first 12: {hits[:12]}")
-print(f"  Sanity: {[h % 5 for h in hits[:12]]}")
-print()
-# ---------------------------------------------------------------------------
-# Demo 4: Manifest-aware threshold CPU - run a real program
-# ---------------------------------------------------------------------------
-print("=" * 64)
-print(" Demo 4: Threshold CPU running an assembled program")
-print("=" * 64)
-class ThresholdCPU10:
-    """CPU runtime matching the bundled small-profile manifest (10-bit addr)."""
-    def __init__(self, addr_bits, mem_bytes):
-        self.addr_bits = addr_bits
-        self.mem_bytes = mem_bytes
-    # --- memory primitives, fully through threshold weights ---
-    def addr_decode(self, addr):
-        bits = torch.tensor(int_to_bits_msb(addr, self.addr_bits), dtype=torch.float32)
-        w = T["memory.addr_decode.weight"]
-        b = T["memory.addr_decode.bias"]
-        return heaviside((w * bits).sum(dim=1) + b)  # [mem_bytes]
-    def mem_read(self, mem, addr):
-        sel = self.addr_decode(addr)
-        mem_bits = torch.tensor(
-            [int_to_bits_msb(byte, 8) for byte in mem], dtype=torch.float32
-        )
-        and_w = T["memory.read.and.weight"]
-        and_b = T["memory.read.and.bias"]
-        or_w = T["memory.read.or.weight"]
-        or_b = T["memory.read.or.bias"]
-        out_bits = []
-        for bit in range(8):
-            inp = torch.stack([mem_bits[:, bit], sel], dim=1)
-            and_out = heaviside((inp * and_w[bit]).sum(dim=1) + and_b[bit])
-            out_bits.append(int(heaviside((and_out * or_w[bit]).sum() + or_b[bit]).item()))
-        return bits_msb_to_int(out_bits)
-    def mem_write(self, mem, addr, value):
-        sel = self.addr_decode(addr)
-        data_bits = torch.tensor(int_to_bits_msb(value, 8), dtype=torch.float32)
-        mem_bits = torch.tensor(
-            [int_to_bits_msb(byte, 8) for byte in mem], dtype=torch.float32
-        )
-        sel_w = T["memory.write.sel.weight"]
-        sel_b = T["memory.write.sel.bias"]
-        nsel_w = T["memory.write.nsel.weight"].squeeze(1)
-        nsel_b = T["memory.write.nsel.bias"]
-        and_old_w = T["memory.write.and_old.weight"]
-        and_old_b = T["memory.write.and_old.bias"]
-        and_new_w = T["memory.write.and_new.weight"]
-        and_new_b = T["memory.write.and_new.bias"]
-        or_w = T["memory.write.or.weight"]
-        or_b = T["memory.write.or.bias"]
-        we = torch.ones_like(sel)
-        sel_inp = torch.stack([sel, we], dim=1)
-        write_sel = heaviside((sel_inp * sel_w).sum(dim=1) + sel_b)
-        nsel = heaviside(write_sel * nsel_w + nsel_b)
-        new_mem = mem[:]
-        for bit in range(8):
-            old = mem_bits[:, bit]
-            data_bit = data_bits[bit].expand(self.mem_bytes)
-            inp_old = torch.stack([old, nsel], dim=1)
-            inp_new = torch.stack([data_bit, write_sel], dim=1)
-            and_old = heaviside((inp_old * and_old_w[:, bit]).sum(dim=1) + and_old_b[:, bit])
-            and_new = heaviside((inp_new * and_new_w[:, bit]).sum(dim=1) + and_new_b[:, bit])
-            or_inp = torch.stack([and_old, and_new], dim=1)
-            new_bit = heaviside((or_inp * or_w[:, bit]).sum(dim=1) + or_b[:, bit])
-            mem_bits[:, bit] = new_bit
-        return [bits_msb_to_int([int(b) for b in mem_bits[i].tolist()]) for i in range(self.mem_bytes)]
-    # --- helper to use threshold ALU functions defined above ---
-    def step(self, state):
-        if state["halted"]:
-            return state
-        s = dict(state)
-        s["mem"] = state["mem"][:]
-        s["regs"] = state["regs"][:]
-        s["flags"] = state["flags"][:]
-        pc = s["pc"]
-        addr_mask = (1 << self.addr_bits) - 1
-        hi = self.mem_read(s["mem"], pc & addr_mask)
-        lo = self.mem_read(s["mem"], (pc + 1) & addr_mask)
-        ir = ((hi & 0xFF) << 8) | (lo & 0xFF)
-        opcode = (ir >> 12) & 0xF
-        rd = (ir >> 10) & 0x3
-        rs = (ir >> 8) & 0x3
-        imm = ir & 0xFF
-        next_pc = (pc + 2) & addr_mask
-        addr16 = None
-        if opcode in (0xA, 0xB, 0xC, 0xD, 0xE):
-            ah = self.mem_read(s["mem"], next_pc)
-            al = self.mem_read(s["mem"], (next_pc + 1) & addr_mask)
-            addr16 = ((ah & 0xFF) << 8) | (al & 0xFF)
-            next_pc = (next_pc + 2) & addr_mask
-        addr10 = (addr16 & addr_mask) if addr16 is not None else None
-        a = s["regs"][rd]
-        b = s["regs"][rs]
-        write = True
-        result = a
         carry = 0
-        if opcode == 0x0:  # ADD
-            result, carry = alu_add(a, b)
-        elif opcode == 0x1:  # SUB
-            result, carry = alu_sub(a, b)
-        elif opcode == 0x7:  # MUL
-            result = alu_mul(a, b)
-        elif opcode == 0x9:  # CMP
-            _r, carry = alu_sub(a, b)
-            z = 1 if _r == 0 else 0
-            n = 1 if (_r & 0x80) else 0
-            s["flags"] = [z, n, carry, 0]
-            write = False
-            opcode_was_cmp = True
-        elif opcode == 0xA:  # LOAD
-            result = self.mem_read(s["mem"], addr10)
-        elif opcode == 0xB:  # STORE
-            s["mem"] = self.mem_write(s["mem"], addr10, b & 0xFF)
-            write = False
-        elif opcode == 0xC:  # JMP
-            s["pc"] = addr10
-            return s
-        elif opcode == 0xD:  # Jcc
-            cond = imm & 0x7
-            take = False
-            z, n, c, v = s["flags"]
-            if cond == 0: take = z == 1
-            elif cond == 1: take = z == 0
-            elif cond == 2: take = c == 1
-            elif cond == 3: take = c == 0
-            elif cond == 4: take = n == 1
-            elif cond == 5: take = n == 0
-            elif cond == 6: take = v == 1
-            else: take = v == 0
-            s["pc"] = addr10 if take else next_pc
-            return s
-        elif opcode == 0xF:  # HALT
-            s["halted"] = True
-            return s
-        if write and opcode != 0x9:
-            s["regs"][rd] = result & 0xFF
-        if opcode in (0x0, 0x1, 0x7):
-            z = 1 if (result & 0xFF) == 0 else 0
-            n = 1 if (result & 0x80) else 0
-            s["flags"] = [z, n, carry, 0]
-        s["pc"] = next_pc
-        return s
-    def run(self, state, max_cycles=64):
-        s = state
-        cycles = 0
-        while not s["halted"] and cycles < max_cycles:
-            s = self.step(s)
-            cycles += 1
-        return s, cycles
-def encode_instr(opcode, rd, rs, imm):
-    return ((opcode & 0xF) << 12) | ((rd & 0x3) << 10) | ((rs & 0x3) << 8) | (imm & 0xFF)
-def write_word(mem, addr, value):
-    mem[addr] = (value >> 8) & 0xFF
-    mem[addr + 1] = value & 0xFF
-# Program: count down from 5 to 0 with a loop, accumulate sum into R0.
-#
-# R1 = 5
-# R0 = 0
-# loop:
-#   R0 = R0 + R1     ; ADD R0, R1
-#   R1 = R1 - 1      ; we need an immediate decrement; use SUB R1, R2 with R2=1
-#   CMP R1, R3       ; R3=0
-#   JNZ loop
-# HALT
-#
-# Memory layout (1KB):
-#   0x0000: LOAD R1 <- M[0x0100]   (5)
-#   0x0004: LOAD R2 <- M[0x0101]   (1)
-#   0x0008: LOAD R3 <- M[0x0102]   (0)
-#   0x000C: LOAD R0 <- M[0x0102]   (0)
-#   0x0010: ADD  R0, R1
-#   0x0012: SUB  R1, R2
-#   0x0014: CMP  R1, R3
-#   0x0016: JNZ  0x0010
-#   0x001A: STORE R0 -> M[0x0103]
-#   0x001E: HALT
-mem = [0] * 1024
-mem[0x100] = 5
-mem[0x101] = 1
-mem[0x102] = 0
-# LOAD R1 <- M[0x0100]
-write_word(mem, 0x0000, encode_instr(0xA, 1, 0, 0)); write_word(mem, 0x0002, 0x0100)
-# LOAD R2 <- M[0x0101]
-write_word(mem, 0x0004, encode_instr(0xA, 2, 0, 0)); write_word(mem, 0x0006, 0x0101)
-# LOAD R3 <- M[0x0102]
-write_word(mem, 0x0008, encode_instr(0xA, 3, 0, 0)); write_word(mem, 0x000A, 0x0102)
-# LOAD R0 <- M[0x0102]
-write_word(mem, 0x000C, encode_instr(0xA, 0, 0, 0)); write_word(mem, 0x000E, 0x0102)
-# ADD R0, R1
-write_word(mem, 0x0010, encode_instr(0x0, 0, 1, 0))
-# SUB R1, R2
-write_word(mem, 0x0012, encode_instr(0x1, 1, 2, 0))
-# CMP R1, R3
-write_word(mem, 0x0014, encode_instr(0x9, 1, 3, 0))
-# JNZ 0x0010 (cond=1 = NZ)
-write_word(mem, 0x0016, encode_instr(0xD, 0, 0, 0x01)); write_word(mem, 0x0018, 0x0010)
-# STORE R0 -> M[0x0103]
-write_word(mem, 0x001A, encode_instr(0xB, 0, 0, 0)); write_word(mem, 0x001C, 0x0103)
-# HALT
-write_word(mem, 0x001E, encode_instr(0xF, 0, 0, 0))
-cpu = ThresholdCPU10(addr_bits=ADDR_BITS, mem_bytes=MEM_BYTES)
-state = {
-    "pc": 0,
-    "regs": [0, 0, 0, 0],
-    "flags": [0, 0, 0, 0],
-    "mem": mem,
-    "halted": False,
-}
-print("  Program: sum 5+4+3+2+1 via loop (uses ADD/SUB/CMP/Jcc/LOAD/STORE/HALT, all threshold-gated)")
-print("  Running ...")
-final, cycles = cpu.run(state, max_cycles=200)
-print(f"  Halted after {cycles} cycles")
-print(f"  R0={final['regs'][0]}  R1={final['regs'][1]}  R2={final['regs'][2]}  R3={final['regs'][3]}")
-print(f"  M[0x0103] = {final['mem'][0x103]}  (expected 15)")
-print()
-print("Done.")

 """
 Hands-on playground for the 8bit-threshold-computer.
+Loads a safetensors model, reads its manifest, and exercises threshold
+circuits at every level: raw Boolean gates, 8-bit ALU arithmetic and
+comparators, multi-layer modular arithmetic, and a manifest-sized CPU
+runtime running a small assembled program end-to-end through the
+threshold weights.
+The CPU demo defaults to the small (1 KB) profile so the run finishes in
+a fraction of a second. Larger profiles (4 KB, 64 KB) take proportionally
+longer because every memory access decodes against every address line.
+Usage:
+    python play.py                                  # fast 1KB demo
+    python play.py --model neural_computer.safetensors   # full 64KB
+    python play.py --model variants/neural_alu8.safetensors --skip-cpu  # ALU only
 """
 from __future__ import annotations
+import argparse
 import os
 import sys
 import torch
 from safetensors import safe_open
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+# Reuse the variant-aware CPU runtime from eval_all.py
+from eval_all import GenericThresholdCPU, builtin_program
 def heaviside(x):
     return out
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Threshold computer playground")
+    parser.add_argument(
+        "--model", type=str,
+        default=os.path.join(os.path.dirname(__file__),
+                             "variants", "neural_computer8_small.safetensors"),
+        help="Path to a .safetensors variant"
+    )
+    parser.add_argument("--skip-cpu", action="store_true",
+                        help="Skip the CPU program demo (useful for pure-ALU files)")
+    args = parser.parse_args()
+    print("Loading", args.model)
+    T = load_tensors(args.model)
+    DATA_BITS = int(T["manifest.data_bits"].item())
+    ADDR_BITS = int(T["manifest.addr_bits"].item())
+    MEM_BYTES = int(T["manifest.memory_bytes"].item())
+    REGISTERS = int(T["manifest.registers"].item())
+    print(f"Manifest: data={DATA_BITS}-bit, addr={ADDR_BITS}-bit, mem={MEM_BYTES}B, regs={REGISTERS}")
+    print(f"Tensors: {len(T):,}")
+    print(f"Total params: {sum(t.numel() for t in T.values()):,}")
+    print()
+    def gate(name, inputs):
+        w = T[name + ".weight"].view(-1)
+        b = T[name + ".bias"].view(-1)
+        return int(heaviside((torch.tensor(inputs, dtype=torch.float32) * w).sum() + b).item())
+    def xor(prefix, inputs):
+        a, b_ = inputs
+        h_or = gate(f"{prefix}.layer1.or", [a, b_])
+        h_nand = gate(f"{prefix}.layer1.nand", [a, b_])
+        return gate(f"{prefix}.layer2", [h_or, h_nand])
+    def xor_neuron(prefix, inputs):
+        a, b_ = inputs
+        h1 = gate(f"{prefix}.layer1.neuron1", [a, b_])
+        h2 = gate(f"{prefix}.layer1.neuron2", [a, b_])
+        return gate(f"{prefix}.layer2", [h1, h2])
+    def int_to_bits_msb(v, n):
+        return [(v >> (n - 1 - i)) & 1 for i in range(n)]
+    def bits_msb_to_int(bits):
+        out = 0
+        for b in bits:
+            out = (out << 1) | int(b)
+        return out
+    # ---------- Demo 1: Boolean gates ----------
+    print("=" * 64)
+    print(" Demo 1: Boolean threshold gates")
+    print("=" * 64)
+    truth_2 = [(0, 0), (0, 1), (1, 0), (1, 1)]
+    for gname in ["and", "or", "nand", "nor", "implies"]:
+        row = " ".join(f"{a}{b}->{gate(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
+        print(f"  {gname:8} {row}")
+    for gname in ["xor", "xnor", "biimplies"]:
+        row = " ".join(f"{a}{b}->{xor_neuron(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
+        print(f"  {gname:8} {row}")
+    print(f"  not      0->{gate('boolean.not', [0])} 1->{gate('boolean.not', [1])}")
+    print()
+    # ---------- Demo 2: 8-bit ALU arithmetic ----------
+    print("=" * 64)
+    print(" Demo 2: 8-bit ALU arithmetic (every gate is threshold logic)")
+    print("=" * 64)
+    def fa(prefix, a, b, cin):
+        s1 = xor(f"{prefix}.ha1.sum", [a, b])
+        c1 = gate(f"{prefix}.ha1.carry", [a, b])
+        s2 = xor(f"{prefix}.ha2.sum", [s1, cin])
+        c2 = gate(f"{prefix}.ha2.carry", [s1, cin])
+        return s2, gate(f"{prefix}.carry_or", [c1, c2])
+    def alu_add(a, b):
+        a_lsb = list(reversed(int_to_bits_msb(a, 8)))
+        b_lsb = list(reversed(int_to_bits_msb(b, 8)))
         carry = 0
+        sum_lsb = []
+        for i in range(8):
+            s, carry = fa(f"arithmetic.ripplecarry8bit.fa{i}", a_lsb[i], b_lsb[i], carry)
+            sum_lsb.append(s)
+        return bits_msb_to_int(list(reversed(sum_lsb))), carry
+    def alu_sub(a, b):
+        a_lsb = list(reversed(int_to_bits_msb(a, 8)))
+        b_lsb = list(reversed(int_to_bits_msb(b, 8)))
+        carry = 1
+        diff_lsb = []
+        for i in range(8):
+            notb = gate(f"arithmetic.sub8bit.notb{i}", [b_lsb[i]])
+            x1 = xor(f"arithmetic.sub8bit.fa{i}.xor1", [a_lsb[i], notb])
+            x2 = xor(f"arithmetic.sub8bit.fa{i}.xor2", [x1, carry])
+            and1 = gate(f"arithmetic.sub8bit.fa{i}.and1", [a_lsb[i], notb])
+            and2 = gate(f"arithmetic.sub8bit.fa{i}.and2", [x1, carry])
+            carry = gate(f"arithmetic.sub8bit.fa{i}.or_carry", [and1, and2])
+            diff_lsb.append(x2)
+        return bits_msb_to_int(list(reversed(diff_lsb))), carry
+    def alu_compare(a, b, kind):
+        inp = int_to_bits_msb(a, 8) + int_to_bits_msb(b, 8)
+        if kind == "eq":
+            h_geq = gate("arithmetic.equality8bit.layer1.geq", inp)
+            h_leq = gate("arithmetic.equality8bit.layer1.leq", inp)
+            return gate("arithmetic.equality8bit.layer2", [h_geq, h_leq])
+        return gate(f"arithmetic.{kind}8bit", inp)
+    def alu_mul(a, b):
+        a_bits = int_to_bits_msb(a, 8)
+        b_bits = int_to_bits_msb(b, 8)
+        result = 0
+        for j in range(8):
+            if b_bits[j] == 0:
+                continue
+            row = 0
+            for i in range(8):
+                pp = gate(f"alu.alu8bit.mul.pp.a{i}b{j}", [a_bits[i], b_bits[j]])
+                row |= (pp << (7 - i))
+            shift = 7 - j
+            result, _ = alu_add(result & 0xFF, (row << shift) & 0xFF)
+        return result & 0xFF
+    cases_arith = [(5, 3), (37, 100), (200, 99), (255, 1), (127, 128), (15, 17)]
+    print("ADD:")
+    for a, b in cases_arith:
+        r, c = alu_add(a, b)
+        e = (a + b) & 0xFF
+        print(f"  {a:3} + {b:3} = {r:3} (carry={c})  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print("SUB:")
+    for a, b in cases_arith:
+        r, c = alu_sub(a, b)
+        e = (a - b) & 0xFF
+        print(f"  {a:3} - {b:3} = {r:3} (no_borrow={c})  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print("CMP:")
+    for a, b in [(50, 30), (30, 50), (77, 77), (255, 0), (0, 255), (128, 127)]:
+        gt = alu_compare(a, b, "greaterthan")
+        lt = alu_compare(a, b, "lessthan")
+        eq = alu_compare(a, b, "eq")
+        print(f"  {a:3} vs {b:3} -> GT={gt} LT={lt} EQ={eq}")
+    print("MUL (low 8 bits):")
+    for a, b in [(12, 11), (15, 17), (8, 32), (200, 3), (0, 99), (1, 255)]:
+        r = alu_mul(a, b)
+        e = (a * b) & 0xFF
+        print(f"  {a:3} * {b:3} = {r:3}  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print()
+    # ---------- Demo 3: mod-5 divisibility ----------
+    print("=" * 64)
+    print(" Demo 3: mod-5 divisibility (multi-layer, hand-constructed)")
+    print("=" * 64)
+    def mod5(v):
+        bits = int_to_bits_msb(v, 8)
+        n = 0
+        while f"modular.mod5.layer1.geq{n}.weight" in T:
+            n += 1
+        eqs = []
+        for i in range(n):
+            h_geq = gate(f"modular.mod5.layer1.geq{i}", bits)
+            h_leq = gate(f"modular.mod5.layer1.leq{i}", bits)
+            eqs.append(gate(f"modular.mod5.layer2.eq{i}", [h_geq, h_leq]))
+        return gate("modular.mod5.layer3.or", eqs)
+    hits = [v for v in range(256) if mod5(v)]
+    print(f"  v in [0,255] with mod5(v)==1: {len(hits)} hits, first 12: {hits[:12]}")
+    print(f"  Sanity (each %5): {[h % 5 for h in hits[:12]]}")
+    print()
+    # ---------- Demo 4: CPU running an assembled program ----------
+    if args.skip_cpu or MEM_BYTES < 0x84:
+        if args.skip_cpu:
+            print("Demo 4 skipped (--skip-cpu).")
+        else:
+            print(f"Demo 4 skipped (memory={MEM_BYTES}B too small for the demo program).")
+        return 0
+    print("=" * 64)
+    print(f" Demo 4: Threshold CPU running an assembled program ({MEM_BYTES} B memory)")
+    print("=" * 64)
+    print("  Program: sum 5+4+3+2+1 via loop")
+    print("           uses LOAD/STORE/ADD/SUB/CMP/JNZ/HALT, all threshold-gated")
+    print("  Running ... (larger memories take longer because every memory access")
+    print("              decodes against every address line)")
+    cpu = GenericThresholdCPU({k: v for k, v in T.items()})
+    mem, expected = builtin_program(ADDR_BITS)
+    state = {"pc": 0, "regs": [0] * 4, "flags": [0] * 4, "mem": mem, "halted": False}
+    final, cycles = cpu.run(state, max_cycles=200)
+    got = final["mem"][0x83]
+    print(f"  Halted after {cycles} cycles")
+    print(f"  R0={final['regs'][0]} R1={final['regs'][1]} "
+          f"R2={final['regs'][2]} R3={final['regs'][3]}")
+    print(f"  M[0x0083] = {got}  (expected {expected})  [{'OK' if got == expected else 'FAIL'}]")
+    return 0 if got == expected else 1
+if __name__ == "__main__":
+    sys.exit(main())