--- license: mit tags: - threshold-logic - neuromorphic - computer-architecture - turing-complete - recurrent-neural-network - in-memory-computing - self-replication - loihi - truenorth - akida --- # threshold-computers A family of Turing-complete machines implemented entirely as threshold logic gates. Every gate, from Boolean primitives to arithmetic to control flow, is a single threshold neuron of the form: ``` output = 1 if (Σ wᵢ·xᵢ + b) ≥ 0 else 0 ``` **Every weight in the file is in {-1, 0, 1}.** Biases are integers. Activations are the Heaviside step. Nothing else. The library was originally built with positional weights up to ±2³¹ for wide single-layer comparators; those have all been replaced with bit-cascaded multi-layer equivalents that use only ternary weights and small integer biases. Threshold-gate evaluation reduces to a popcount minus a popcount plus a bias, which is exactly what neuromorphic chips and FPGAs natively support. The repository ships eighteen prebuilt configurations spanning three data-path widths (8, 16, 32 bits) and six memory sizes (0 B to 64 KB). The canonical file at the repo root is the largest of these: a 32-bit data path with a 64 KB address space and ~8.61 M tensor elements (~8.46 M gate weights and biases; the rest is `.inputs` routing metadata). ``` neural_computer.safetensors 32-bit data, 64 KB memory, ~8.61M params (canonical) variants/neural_computer{8,16,32}.safetensors full memory (64 KB) variants/neural_computer{8,16,32}_reduced.safetensors 4 KB memory variants/neural_computer{8,16,32}_small.safetensors 1 KB memory variants/neural_computer{8,16,32}_scratchpad.safetensors 256 B memory variants/neural_computer{8,16,32}_registers.safetensors 16 B memory variants/neural_alu{8,16,32}.safetensors pure ALU, no memory variants/neural_subleq8.safetensors one-instruction machine (SUBLEQ) variants/neural_rv32.safetensors RV32IM + F-subset RISC-V processor variants/neural_matrix8.safetensors the CPU as one recurrent ternary matrix stack variants/neural_subleq8io.safetensors SUBLEQ host for the universal constructor variants/neural_reflect.safetensors interpreter whose state holds its own weights variants/neural_attractor.safetensors energy-based solver; a multiplier run backward factors variants/neural_reversible.safetensors reversible arithmetic core, a bijection with no erasure variants/neural_ca.safetensors reversible cellular-automaton medium (no processor) variants/neural_tile.safetensors self-assembling tile computer (computation as growth) ``` Nine further machines are detailed in their own sections below, and together they carry the family from the smallest possible processor to several results about what a threshold network can be. `neural_subleq8` is a Turing-complete one-instruction computer whose entire control flow is a single threshold neuron. `neural_rv32` is a RISC-V processor (RV32IM plus an F subset) that runs stock-compiler C, with dual-issue execution, memory-mapped I/O, and a self-referential NEUR opcode. `neural_matrix8` drops the gate graph entirely: the whole processor is compiled into a fixed stack of ternary weight matrices with a Heaviside step between them, so one clock cycle is one matrix-vector product followed by thresholding, exactly what a resistive or photonic crossbar computes. `neural_subleq8io` hosts a universal constructor: a program that reads a description of any machine in the family and prints that machine's weight file byte for byte, self-reproduction being the case where the description is its own. And `neural_reflect` is a universal interpreter over ternary threshold netlists: its fixed transition circuit evaluates a netlist held in the writable state, so the machine it runs is stored data that a program reads, edits, and reproduces. `neural_attractor` drops sequential control entirely: a circuit compiles to an energy function whose minimum is its consistent assignment, so clamping different wires runs the same network forward to evaluate, backward to invert (a multiplier run backward returns factors), or as a SAT solver. And `neural_reversible` makes the entire state transition a bijection, so no step erases information and the machine runs backward to reconstruct its input, a processor with no Landauer erasure floor. And `neural_ca` has no processor at all: one fixed reversible rule applied to every 2x2 block of a lattice (a Margolus cellular automaton), where a particle collision computes an AND gate and ballistic transport plus collisions are the billiard-ball universality primitives. And `neural_tile` computes by self-assembly: a tile set whose binding rule is a threshold gate grows a crystal that is the trace of a computation, a Sierpinski (Rule 90) pattern or a binary counter, Turing-universal at temperature 2 (Winfree 1998). --- ## Quick start ```python import torch from safetensors.torch import load_file tensors = load_file("neural_computer.safetensors") def heaviside(x): return (x >= 0).float() # AND gate: fires when both inputs are 1 w = tensors['boolean.and.weight'] # [2] b = tensors['boolean.and.bias'] # [1] for a, c in [(0, 0), (0, 1), (1, 0), (1, 1)]: out = heaviside((torch.tensor([a, c], dtype=torch.float32) * w).sum() + b) print(f"AND({a}, {c}) = {int(out.item())}") ``` Run the full circuit verification suite against any variant: ```bash python src/eval_all.py variants/ # 18 fitness variants (machines skipped) python src/eval_all.py neural_computer.safetensors # the canonical file python src/eval_all.py --cpu-program variants/ # also run an assembled # program through the # threshold-gated CPU ``` `src/eval_all.py` reads each variant's manifest, runs a gate-level fitness suite (13,900–15,900 tests per variant covering Boolean, arithmetic, ALU, control, modular, error-detection, threshold, and float circuits, including end-to-end evaluation of the composed float pipelines from the shipped wiring metadata — see the Verification table), and optionally executes a small assembled program through a manifest-sized threshold CPU plus a chained 16- or 32-bit ALU sequence on wider variants. For an interactive walkthrough that exercises Boolean gates, the 8-bit ALU, mod-5 divisibility, and a CPU loop end-to-end: ```bash python tools/play.py # 1 KB demo, runs in seconds python tools/play.py --model neural_computer.safetensors # 64 KB, slower ``` For end-to-end CPU validation (Fibonacci, sum 1..N, bubble sort, self-modifying JMP, all eight conditional jumps, CALL stack semantics, MUL cross-checked against repeated ADD, DIV cross-checked against repeated SUB, a bitwise AND/OR/XOR/SHL/SHR chain, and the architectural flag policy): ```bash python tools/test_cpu.py # default: 1 KB, ~2 s python tools/test_cpu.py --model neural_computer.safetensors # 64 KB canonical, ~2 min python tools/test_cpu.py --only fib,sum_n # subset of suite ``` Each program is assembled by a small Python assembler (`tools/cpu_programs.py`) and run through the threshold-gated CPU; the driver verifies expected memory contents at HALT. --- ## Execution model A self-contained machine. State goes in, state comes out: - **Pure tensor computation**: state in, state out - **Frozen circuits**: integer weights, Heaviside activation - **ACT execution**: internal loop until `HALT` - **No external orchestration**: one forward pass equals one complete program execution Every datapath operation runs through threshold gates: ALU arithmetic (ADD/SUB via ripple-carry, MUL via partial-product AND gates and shift-add, DIV via per-stage bit-cascade comparators), bitwise logic, shifts, comparisons, conditional-jump PC muxing, stack-pointer stepping, and all memory reads and writes. Instruction decode (a 4-to-16 opcode one-hot decoder) and PC sequencing (the PC+2/PC+4 increment chains and the next-PC mux) are also threshold gates now; register-field indexing and the bit complements feeding negated-condition selects remain structural routing. ``` ┌─────────────────────────────┐ │ Initial State │ │ [PC|Regs|Flags|Memory...] │ └─────────────┬───────────────┘ ▼ ┌─────────────────────────────┐ │ Threshold Circuit Layer │ │ ┌───────────────────────┐ │ │ │ Fetch: PC → Instr │ │ │ ├───────────────────────┤ │ │ │ Decode: Opcode/Ops │ │ │ ├───────────────────────┤ │ │ │ Execute: ALU/Mem │ │ │ ├───────────────────────┤ │ │ │ Writeback: Results │ │ │ ├───────────────────────┤ │ │ │ PC Update │ │ │ └───────────┬───────────┘ │ │ │ │ │ ┌────▼────┐ │ │ │ HALTED? │ │ │ └────┬────┘ │ │ no ────┴──── yes │ │ │ │ │ │ ▼ ▼ │ │ [loop] [exit] │ └─────────────┬───────────────┘ ▼ ┌─────────────────────────────┐ │ Final State │ └─────────────────────────────┘ ``` ### Instruction set | Opcode | Mnemonic | Operation | |--------|----------|-----------| | 0x0 | ADD | R[d] = R[a] + R[b] | | 0x1 | SUB | R[d] = R[a] - R[b] | | 0x2 | AND | R[d] = R[a] & R[b] | | 0x3 | OR | R[d] = R[a] \| R[b] | | 0x4 | XOR | R[d] = R[a] ^ R[b] | | 0x5 | SHL | R[d] = R[a] << 1 | | 0x6 | SHR | R[d] = R[a] >> 1 | | 0x7 | MUL | R[d] = R[a] * R[b] | | 0x8 | DIV | R[d] = R[a] / R[b] | | 0x9 | CMP | flags = R[a] - R[b] | | 0xA | LOAD | R[d] = M[addr] | | 0xB | STORE | M[addr] = R[s] | | 0xC | JMP | PC = addr | | 0xD | Jcc | PC = addr if cond (imm8[2:0]: 0=Z, 1=NZ, 2=C, 3=NC, 4=N, 5=P, 6=V, 7=NV) | | 0xE | CALL | push PC; PC = addr | | 0xF | HALT | stop execution | **Flag policy.** Only ADD, SUB, MUL, and CMP write the FLAGS register; AND, OR, XOR, SHL, SHR, DIV, LOAD, STORE, and all control transfers leave it unchanged. Branch on the arithmetic op that set the condition — intervening bitwise or shift instructions do not disturb it. The `flags_policy` program in the CPU suite pins this behavior. | Flag | ADD | SUB / CMP | MUL | |---|---|---|---| | Z | result == 0 | result == 0 | result == 0 | | N | result bit 7 | result bit 7 | result bit 7 | | C | carry-out | 1 when no borrow (`a >= b`) | 0 | | V | signed overflow | signed overflow | 0 | ### State tensor layout The **state tensor** uses MSB-first bit ordering: index 0 of each multi-bit field is the most-significant bit. So `R0[0]` is bit 7 of the architectural register, `R0[7]` is bit 0. ``` [ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ] ``` `N` is the address width (configurable, 0–16). Flags are ordered `Z, N, C, V`. Control bits are ordered `HALT, MEM_WE, MEM_RE, RESERVED`. #### Bit ordering, one rule per scope The state tensor's MSB-first convention does **not** propagate to subcircuit ports. Each subcircuit names its operand bits in its own scope: | Scope | Convention | Example | |---|---|---| | State tensor | MSB-first (index 0 = MSB) | `R0[0]` is bit 7 of register R0 | | Integer subcircuit ports (`$a[i]`, `$b[i]`) | LSB-indexed (index 0 = LSB) | `$a[0]` is bit 0 of operand `a` | | Ripple-carry full adders (`fa0..fa7`) | LSB-first (fa0 = LSB) | `fa0` consumes `$a[0]` and `$b[0]` | | Composed float circuits (`float16.add` etc.) | MSB-first operand words | `$a[0]` is the sign bit, `$a[1..E]` the exponent | | Float result gates (`exp_out.bit{k}`, `frac_out.bit{k}`) | LSB-first field bits | `frac_out.bit0` is the fraction LSB | | Instruction word | MSB-first (bit 15 = opcode high) | bit 15 is `opcode[3]` | Worked example for `arithmetic.ripplecarry8bit`: - Inputs: `$a[0]..$a[7]` and `$b[0]..$b[7]` where `$a[0]` is the LSB of `a`. To add `a = 0x05 = 0b00000101` and `b = 0x03`, drive `a[0]=1, a[1]=0, a[2]=1` (rest 0) and `b[0]=1, b[1]=1` (rest 0). - Outputs: `fa0.ha2.sum.layer2`..`fa7.ha2.sum.layer2` are sum bits 0..7 (LSB to MSB), and `fa7.carry_or` is the final carry-out. The 8-bit result is `{fa7..fa0}` reading high-to-low. This is also how `safetensors2verilog`'s threshold-logic frontend exposes the ports of any extracted subcircuit; `python -m safetensors2verilog ... --inspect` prints the port contract for any extracted circuit. ### Instruction encoding (16-bit, MSB-first) ``` 15..12 11..10 9..8 7..0 opcode rd rs imm8 ``` Interpretation: - **R-type**: `rd = rd op rs` (imm8 ignored) - **I-type**: `rd = op rd, imm8` (rs ignored) - **Address-extended**: `LOAD`, `STORE`, `JMP`, `Jcc`, `CALL` consume the next word as a 16-bit address (big-endian); `imm8` is reserved and the PC skips 4 bytes when the jump is not taken. ### Circuit categories | Category | Circuits | Examples | |----------|----------|----------| | Boolean | 9 | AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES | | Arithmetic | 18+ | half/full adder, ripple-carry (8/16/32-bit), comparators (8/16/32-bit), 3-operand adder, A+B×C and (A+B)×C expressions | | ALU | 8/16/32-bit | shifts, multiply, divide, INC/DEC, NEG, ROL/ROR, bitwise | | Combinational | 10+ | MUX (2:1, 4:1, 8:1), DEMUX, 3-to-8 decoder, 8-to-3 encoder, barrel shifter, priority encoder | | Control flow | 16 | JMP, conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV), CALL, RET, PUSH, POP | | Memory | 3 | N-bit address decoder, read mux, write cells (packed) | | Modular | 11 | divisibility by 2–12 (multi-layer for non-powers-of-2) | | Threshold | 13 | k-of-n gates, majority, minority, exactly-k | | Pattern | 10 | popcount, leading/trailing ones, symmetry | | Error detection | 11 | parity (XOR tree), checksum, CRC, Hamming | | Float (IEEE 754) | half + single | composed ADD, MUL, DIV, EQ/LT/LE/GT/GE pipelines plus unpack/pack/classify/normalize stages | The float ADD/MUL/DIV/CMP circuits are **self-contained composed pipelines**: their complete internal wiring ships in the files as `.inputs` metadata, their external inputs are the raw operand words (`$a[0]` sign, `$a[1..E]` exponent, then mantissa; MSB-first), and the eval suite reconstructs and executes each netlist end to end from that metadata alone (`NetlistEvaluator`). Comparisons are fully IEEE (NaN unordered, `+0 == -0`, subnormal ordering, mixed signs). The arithmetic is round-to-nearest-even, **bit-exact to IEEE hardware** including the subnormal range, with exact specials (NaN, infinities including `inf - inf`/`inf * 0`/`0/0`/`inf/inf` → NaN and `x/0` → inf, signed zeros), subnormal operands, and gradual-underflow subnormal results (each oracle validated bit-exact against numpy over every class). Because the tests run from each file's own routing metadata, they prove the artifact is self-contained: `safetensors2verilog`-style extraction of `float16.add` and friends has everything it needs to emit a single composed module. ### Tensor naming ``` {category}.{circuit}[.{layer}][.{component}].{weight|bias} Examples: boolean.and.weight boolean.xor.layer1.neuron1.weight arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight modular.mod5.eq.k15.bit3.match.weight error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias ``` Memory circuits are stored as packed tensors so the safetensors header stays manageable (`memory.addr_decode.weight`, `memory.read.and.weight`, `memory.write.and_old.weight`, etc.). --- ## Bit widths and memory profiles The build tool emits one of 51 functionally distinct configurations: three data-path widths × seventeen address widths (0–16, where 0 means no memory). **Bit widths** (`--bits`): | Width | Range | Use case | |-------|-------|----------| | 8 | 0–255 | full CPU, legacy compatibility | | 16 | 0–65,535 | extended arithmetic | | 32 | 0–4,294,967,295 | practical arithmetic ranges | **Memory profiles** (`-m`): | Profile | Size | Addr bits | Filename suffix | |---------|------|-----------|-----------------| | `none` | 0 B | 0 | (uses `alu` instead of `computer`) | | `registers` | 16 B | 4 | `_registers` | | `scratchpad` | 256 B | 8 | `_scratchpad` | | `small` | 1 KB | 10 | `_small` | | `reduced` | 4 KB | 12 | `_reduced` | | `full` | 64 KB | 16 | (none) | Auto-generated filename: `neural_{alu|computer}{BITS}[_{MEMORY}].safetensors`. Custom address widths via `-a N` produce `_addrN`. ```bash python src/build.py --bits 32 --apply all # neural_computer32.safetensors python src/build.py --bits 8 -m none --apply all # neural_alu8.safetensors python src/build.py --bits 16 -m small --apply all # neural_computer16_small.safetensors python src/build.py --bits 32 -a 6 --apply all # neural_computer32_addr6.safetensors ``` To regenerate every named variant in one pass: ```bash python tools/build_all.py ``` This populates `variants/` with all 18 builds, quantizes each one to the smallest signed integer dtype that exactly represents its weights (~4× reduction in tensor data, with file size dominated by the safetensors header on the smaller profiles), verifies the strictly ternary weight invariant (`--ternary --strict`, so a build with any non-ternary weight fails loudly), stamps the `weight_quantization` metadata field, and runs `src/eval.py` on each as a sanity check. The quantizer is also available standalone: ```bash python src/quantize.py path/to/file.safetensors # in-place python src/quantize.py variants/ # whole directory python src/quantize.py model.safetensors -o quantized.safetensors python src/quantize.py file.safetensors --ternary # push toward {-1, 0, 1} weights python src/quantize.py file.safetensors --ternary --strict # error if any weight is non-ternary ``` Every weight and bias tensor in the canonical model fits in `int8`. The eval pipeline promotes weights to `float32` on load, so integer storage is exact and transparent. **Ternary mode.** `src/build.py` emits only ternary weights: identity buffers are `weight=1, bias=-1` (`H(x - 1)`), and the comparators, modular detectors, and division stages that previously required positional weights up to ±2³¹ are bit-cascaded multi-layer equivalents. With `--ternary`, the quantizer verifies this and repairs legacy files: it rewrites historical single-input `weight=±2` buffers as `weight=±1` with the bias adjusted to preserve the heaviside output for binary inputs (`H(2x - 1) ≡ H(x - 1)`), and rebuilds pre-bit-cascade modular detectors (moduli already in bit-cascade form are left untouched, routing metadata included). `--strict` fails if any weight tensor remains non-ternary. Every shipped file carries the metadata field `weight_quantization: ternary`; a repaired file with remaining violations would be stamped `ternary_partial`. --- ## Verification | Category | Coverage | Notes | |----------|----------|-------| | Boolean gates | exhaustive | all 2^n input combinations | | Arithmetic (8-bit) | strategic sampling | edge values + diagonal pairs; ~50 cases per circuit | | Arithmetic (16/32-bit) | strategic sampling | width-scaled extremes, alternating patterns, byte-boundary carries | | ALU primitives (8/16/32-bit) | strategic sampling | edge inputs per operation; DIV comparators driven along real restoring-division traces | | Control flow | exhaustive | all 2^3 input combinations per Jcc, per address bit | | Threshold k-of-n | exhaustive | all 256 8-bit popcounts | | Modular (all moduli, 8-bit input) | exhaustive | every value in [0, 255] | | Parity | exhaustive | every value in [0, 255] | | Pattern recognition | exhaustive | every value in [0, 255] | | Combinational logic | exhaustive | full input space per gate | | Float unpack/pack | exhaustive, functional | every bit gate driven with 0 and 1 (identity) | | Float classify | functional | IEEE 754 categories (zero, subnormal, normal, inf, NaN) at edge encodings, both widths | | Float CMP (composed) | functional, exact IEEE | full netlist rebuilt from the shipped `.inputs` metadata and evaluated end to end; NaN unordered, signed zeros, subnormal ordering, mixed signs — all five predicates, both widths | | Float ADD/MUL/DIV (composed) | functional, bit-exact to IEEE hardware | same metadata-driven evaluation: exact specials, subnormal operands and gradual-underflow results, round-to-nearest-even; cross-checked against native float arithmetic | | Memory / manifest | structure checks | packed-tensor shapes against the manifest | | CPU integration | program-level | ten assembled programs (Fibonacci, sum, sort, self-modifying JMP, all eight Jcc, CALL stack push, MUL vs repeated ADD, DIV vs repeated SUB, a bitwise AND/OR/XOR/SHL/SHR pipeline, and the flag-policy pin) | The 8-bit arithmetic and ALU tests use strategic sampling rather than the full 65,536-case sweep because exhaustive coverage at 8-bit is feasible but not necessary given that the circuits are constructed gate-by-gate. The 16-bit and 32-bit arithmetic tests sample edge cases only; full exhaustive coverage at those widths is infeasible without specialized hardware. `src/eval_all.py` runs the unified suite. Exit code is the number of failing variants (0 means all pass). **Testing is evaluation, not rebuilding**: `python src/eval_all.py variants/` scores all 18 fitness variants against the shipped weights in about two minutes (~6 s each, the composed float netlists evaluated in `NetlistEvaluator`'s leveled mode) and cleanly skips the nine standalone machines. Rebuilding the models (`tools/build_all.py`, ~50 min for all 18) is a separate step, needed only when the circuit constructions in `src/build.py` change; routine verification never rebuilds. The batched evaluator is population-safe: every chained intermediate (carry, borrow, mux select) is computed per population slot, so `tools/prune_weights.py`'s parallel fitness screens are exact rather than slot-0 approximations. --- ## neural_subleq8 — the one-instruction machine The minimal member of the family: a Turing-complete computer with **no instruction decoder** (there is only one instruction), **no registers**, and **no flags** — 194 logic gates totalling 548 ternary parameters. Its one architectural decision, branch or fall through, is a single threshold neuron (`subleq.leq`) over the sign and zero of the subtraction result. - 8-bit data, 8-bit addresses, 256 bytes of packed threshold memory, a program counter and nothing else. - An instruction is three bytes `A B C`. Step: `M[B] = (M[B] - M[A]) mod 256`; if the result is `<= 0` in two's complement (zero or bit 7 set), `PC = C`, else `PC += 3`. `PC = 0xFF` halts. - Circuits (all wiring shipped as `.inputs` metadata): an 8-bit two's-complement subtractor, the branch neuron, a `PC + 3` incrementer, a branch mux, and the packed 256-row memory. ```bash python src/build.py --apply subleq # -> variants/neural_subleq8.safetensors python src/machines.py subleq # exhaustive datapath + lockstep programs ``` `src/machines.py subleq` evaluates the datapath **from the shipped wiring metadata** over all 65,536 operand pairs (result and branch decision both exhaustive), checks the PC mux, and runs a program suite (clear, negate-copy, add-by-double-negation, countdown loop) in full-state lockstep against a reference emulator. Third-party SUBLEQ toolchains target this machine directly once the `0xFF` halt convention is mapped. --- ## neural_rv32 — a RISC-V processor as a threshold network The most capable member: a RISC-V CPU whose entire datapath is ternary threshold gates (`variants/neural_rv32.safetensors`, ~8.6 M parameters, 64 KB memory, strictly ternary). - **RV32I base**: LUI, AUIPC, JAL, JALR, all six branches, LB/LH/LW/LBU/LHU, SB/SH/SW, and the full OP-IMM/OP groups. 32 × 32-bit registers (x0 zero), little-endian memory through the packed threshold circuits. ECALL halts. - **M extension**: MUL, MULH, MULHSU, MULHU (full 64-bit product through a shift-add array with gate-level sign correction), DIV, DIVU, REM, REMU (32 restoring stages, spec-exact divide-by-zero and overflow). - **F subset**: FLW, FSW, FMV.W.X, FMV.X.W, FADD.S, FSUB.S, FMUL.S, FDIV.S, FMADD.S, FMSUB.S, FNMADD.S, FNMSUB.S, FEQ.S, FLT.S, FLE.S, FSGNJ[N/X].S, FCVT.W.S (honors the instruction's rounding-mode field), FCVT.S.W — the arithmetic executed by the composed float32 pipelines (round-to-nearest-even, bit-exact to hardware; specials and subnormals as above; the fused multiply-add rounds `a*b+c` with a single rounding), the int/float conversions gate-routed through the priority encoder and barrel shifter and cross-checked bit-exact against native conversion. - **NEUR (custom-0)**: `neur rd, rs1, rs2` evaluates one threshold neuron — `rd = H(popcount(rs1[7:0] & rs2[7:0]) - popcount(rs1[7:0] & rs2[15:8]) + sext(rs2[20:16]))`. Networks of NEUR instructions are neural networks running as software on the neural network; the test suite computes XOR with a two-layer NEUR net. - **Dual issue**: two adjacent OP/OP-IMM/LUI/AUIPC instructions retire in one cycle when the gate-level hazard comparators (`rv32.hazard.*`) clear RAW and WAW dependences. - **MMIO**: stores to `0xFF00` append a character to the console. Signed comparisons ride the unsigned bit-cascade with sign bits complemented through NOT gates; SLL uses the barrel shifter, SRL is bit-reversal wiring over it, SRA a gate mux over the complement form. Instruction decode (opcode-class detectors and the I/S/B/U/J immediate mux) and PC sequencing (the next-PC mux over PC+4, PC+imm, and (rs1+imm)&~1) are threshold gates; register-field indexing remains fixed wiring. ```bash python src/build.py --apply rv32 # build the file python src/quantize.py variants/neural_rv32.safetensors --ternary --strict python src/machines.py rv32 # eight-program lockstep suite python src/machines.py rv32-c # stock-compiler C, end to end ``` **Running compiled C.** `machines.py rv32-c` compiles a freestanding C program (gcd, Fibonacci, insertion sort; `rv32im`, so real `mul`/`rem`) with an unmodified clang rv32im toolchain, loads the relocatable object with an in-repo loader (no external linker — it resolves the R_RISCV relocations of one translation unit and lays the sections out flat), executes it on the threshold CPU, and checks the return value against the value computed natively. The program retires in ~300 instructions and matches exactly. Stock `rv32im` toolchains (gcc, clang, rustc) emit this ISA. The composed circuits evaluate in a leveled mode (`NetlistEvaluator`, one padded tensor op per topological level instead of one Python step per gate), ~18× faster on the FPU-scale netlists. --- ## neural_matrix8 — the processor as one recurrent threshold network Every machine above is a graph of individually-named gates. `neural_matrix8` is the same processor with the graph dissolved: the verified gate wiring of the registers-profile CPU (8-bit data, 4-bit addresses, 16 B of memory) is compiled into a fixed sequence of ternary weight matrices `W1..Wk` with a Heaviside step between them, acting on a single state vector that holds the program counter, all four registers, the flags, the stack pointer, the halt bit, and every memory bit: ``` [ PC[4] | R0[8] R1[8] R2[8] R3[8] | Z N C V | SP[4] | HALT | MEM[16][8] ] = 173 bits ``` One instruction is one pass through the stack, `s' = H(Wk · … H(W1·s + b1) … + bk)`; the machine is that stack iterated until the halt bit sets, so the whole processor is a single recurrent linear-threshold network. The transition is total: a halted state is a fixed point (every architectural write is gated by NOT halt), so iterating past HALT is harmless. - **Ternary and small-integer, end to end.** The step circuit is assembled from the family's verified cells (two-layer OR/NAND XOR, ripple-carry full adders, bit-cascade comparators, restoring-division stages, 2:1 muxes, one-gate address-decode rows), then levelized with identity pass-throughs carrying live signals forward. It compiles to 108 matrices whose every entry is in `{-1, 0, 1}` with small integer biases (~9.1 MB on disk). - **Equality is machine-checked bit for bit,** not sampled: all 65,536 `(a, b)` operand pairs for each of the ten ALU opcodes (ADD SUB AND OR XOR SHL SHR MUL DIV CMP) batched through the matrices against an integer reference; the full Jcc × flag, JMP, CALL, and LOAD/STORE control space; 2,000 uniformly random full states plus 500 halted-state fixed points; and per-instruction full-state lockstep against the gate-graph `GenericThresholdCPU` walking the shipped `neural_computer8_registers` weights. - **Analog realization.** Pre-activations are integers; a firing gate sits at `>= 0` and a non-firing gate at `<= -1`, so a comparator threshold at `-0.5` gives every gate a guaranteed noise margin of exactly 0.5 (measured minimum over random states: 0.500). Any total analog error below 0.5 per pre-activation therefore reproduces the digital circuit bit for bit. Each matrix maps to a crossbar with conductances drawn from `{-1, 0, +1}` and one clock cycle is one analog matrix-vector product followed by thresholding; the suite injects Gaussian read noise (bit-exact through σ ≈ 0.10, breaking down near σ = 0.15 exactly where the error model predicts) and static conductance mismatch (bit-exact through σ_G = 0.10). ```bash python src/matrix8.py build # compile + save variants/neural_matrix8.safetensors python src/matrix8.py verify # exhaustive equality vs the gate graph and integer reference python src/matrix8.py analog # margin measurement + read-noise + conductance-mismatch sweeps ``` Because the transition is a fixed matrix stack proven equal to the gate-graph processor, the ISA semantics carry over unchanged, and the digital circuit and its analog crossbar realization coincide within the device's quantization and noise margins. The processor is no longer *described by* a neural network; it *is* one, iterated. --- ## neural_attractor — computation as energy relaxation An energy-based threshold network with no program counter, clock, or instruction stream. A Boolean circuit compiles to a quadratic energy `E(s) = sum_i L[i] s_i + sum_{i (a, b, c, 0, f)` for `f = (a+b) XOR c` with the scratch returned to zero, so the reversible machine computes what the irreversible machines do without erasing. A bijective transition erases no bits and therefore carries no Landauer floor of `kT ln 2` per erased bit. The reversible circuits compile to the same ternary matrix stack `neural_matrix8` uses: the 4-bit adder becomes 39 ternary matrices with a Heaviside step, its composed transition is a verified permutation of the state space, and every pre-activation clears the analog threshold by the same 0.5 margin, so the crossbar realization is bit-exact and information-theoretically lossless, holding under injected read noise through sigma ~ 0.10 and static conductance mismatch through sigma_G ~ 0.10, the same tolerances `neural_matrix8` measures. Turning that into measured energy below the Landauer bound requires adiabatic drive of the crossbar, which is the physical frontier; the logical reversibility and its lossless matrix realization are proven here. The arithmetic core ships as `variants/neural_reversible.safetensors`, an 8-bit in-place adder (`b <- a+b`) stored as its reversible gate sequence together with the Heaviside AND/XOR weights that realize each gate, so the file holds both the reversible program and the threshold substrate it runs on. ```bash python src/reversible.py # reversible gates, Cuccaro ALU, Bennett construction python src/reversible_cpu.py # bijective transition and backward execution python src/reversible_prog.py # structured reversible programs (multiply, Fibonacci, Janus IF) python tools/build_reversible.py # ship + round-trip variants/neural_reversible.safetensors python tools/reversible_matrix.py # ternary matrix stack: permutation transition + 0.5 margin ``` --- ## neural_ca — a reversible cellular automaton A partitioned (Margolus) cellular automaton: one fixed rule applied to every 2x2 block of a lattice, with the block partition alternating each step. The state is the lattice; there is no program counter, register file, or control circuit, and one step updates every block. The block rule rotates each block 180 degrees, except that two particles on a diagonal swap to the other diagonal. It is a permutation of the sixteen block states and its own inverse, so the lattice update is a bijection: replaying the partition sequence in reverse reconstructs any earlier configuration (verified over random lattices), and particle number is conserved. The rule is expressed in the family's Heaviside threshold gates (a diagonal-pair detector XORed onto the rotated cells) and compiles to a 6-layer ternary matrix tile that is a permutation with a 0.5 analog margin, bit-exact under read noise through sigma ~ 0.10 and conductance mismatch through sigma_G ~ 0.10; that tile applied to every block is one lattice step, stored as `variants/neural_ca.safetensors`. The dynamics are the billiard-ball model's: a single particle moves ballistically along a diagonal at one cell per step, and two particles collide and deflect reversibly (verified as a genuine interaction, distinct from independent motion). A collision computes logic directly: with input particles at (2,2) and (7,7), three output cells at step 4 carry A AND B (the deflected paths (3,6) and (6,3)), A AND NOT B ((6,6)), and NOT A AND B ((3,3)), the billiard-ball interaction gate, verified over all four inputs. The gates compose: launching a third particle at (0,9) to collide with the A AND B output makes cell (3,5) carry A AND B AND C, verified over all eight inputs, so chained collisions build larger circuits. With mirror routing and a constant-particle source the construction is functionally complete (Margolus 1984). ```bash python src/ca.py # rule bijection, lattice reversibility, ballistic motion, interaction gate python tools/build_ca.py # ship the block rule as a ternary matrix tile (permutation + 0.5 margin) ``` --- ## neural_tile — computation as self-assembly A tile computer in the abstract tile assembly model. The program is a finite set of square tiles with glue labels and integer strengths on their edges; a seed is placed and tiles accrete onto the assembly by one rule: a tile binds at an empty site when the summed strength of its glues that match the already-present neighbors reaches the temperature tau. That binding decision is the Heaviside gate `H(sum_d strength_d * match_d - tau)`, weights the glue strengths and bias `-tau`, so every attachment is a threshold neuron and the assembly grows site by site under the family's gate. At tau = 2 the model is Turing-universal (Winfree 1998). Two directed tile sets are verified. The rule-tile set computes `value(x,y) = f(value(x-1,y), value(x,y-1))` for any 2-input `f`: with `f` = XOR the assembly is the Sierpinski triangle (Rule 90), and AND and OR grow their own patterns, each of 529 tiles checked cell by cell against the recurrence. The binary counter grows one integer per row: at 8-bit width it fills 255 rows and row y encodes the integer y, with the increment carry propagating westward by cooperative binding. Both tile sets are directed, so a unique tile binds at each site and the assembly is deterministic. The shipped `variants/neural_tile.safetensors` stores the counter tile set as its glue tables together with the per-tile binding-gate weights (glue strengths) and bias (`-tau`); reloading it regrows the counter and reproduces the binding decision from the gate. ```bash python src/tile.py # binding gate, XOR/AND/OR rule tiles, binary counter python tools/build_tile.py # ship + regrow variants/neural_tile.safetensors ``` --- ## Threshold logic A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing the result to a threshold; the output is 1 when the sum meets or exceeds the threshold and 0 otherwise. Equivalently, it is a neuron with Heaviside step activation, integer weights, and an integer bias. Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function, which includes AND, OR, NAND, NOR, IMPLIES, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (XOR, parity, mod-k for k not a power of two) require multiple layers. Example gates: ``` AND: w=[1, 1], b=-2 H(0+0-2) = 0 H(1+1-2) = 1 OR: w=[1, 1], b=-1 H(0+0-1) = 0 H(1+0-1) = 1 XOR: two layers (not linearly separable) layer 1: OR + NAND layer 2: AND of the two ``` A full adder is two half-adders plus a carry OR, around four threshold layers. An 8-bit ripple-carry adder is eight chained full adders, around 32 layers. ### History Warren McCulloch and Walter Pitts introduced the threshold neuron in 1943, proving that networks of such neurons can compute any Boolean function. Their work preceded both the perceptron and modern neural networks and established the theoretical foundation for neural computation. The 1960s saw substantial work on threshold logic synthesis. Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function can be implemented as a single threshold gate, and if so, how to compute the appropriate weights. The focus was on individual gates rather than complete systems. Frank Rosenblatt's Mark I Perceptron (1957–1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training; the final configurations were not published. Bernard Widrow's ADALINE and MADALINE (1960–1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm. Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing-complete. The construction relied on continuous sigmoid activations with infinite precision, not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: universality with continuous activations chosen to support gradient-based training. ### Neuromorphic hardware Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon: - **Intel Loihi** (2017): 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. Supports integer weights and configurable neuron dynamics. - **IBM TrueNorth** (2014): one million neurons and 256 million synapses across a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds. - **BrainChip Akida** (2021): edge-oriented event-based processing with integer weights. - **SpiNNaker** (University of Manchester): ARM cores simulating spiking networks at scale. Published work on these platforms has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1 and described that work as "a first step toward the construction of a spiking computer"; that implementation lacked instruction fetch, a program counter, memory, and control logic. The weights in this repository implement a complete CPU: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every logic component is a threshold neuron with integer weights; opcode decode and PC sequencing run through threshold gates as well, leaving only register-field indexing as structural wiring. --- ## Hardware compatibility All weights are in {-1, 0, 1}, all activations are Heaviside step, and every gate is a single weighted sum followed by a sign test. This eliminates multipliers entirely: each gate evaluation is a popcount of `+1`-weighted inputs minus a popcount of `-1`-weighted inputs plus an integer bias. The circuits are intended to deploy directly on: - **FPGA**: every gate maps to a small LUT cluster (or a popcount tree of LUT4/LUT6 + carry chain). Ternary weight storage compresses to 2 bits per weight; routing collapses to bit selection. - **Intel Loihi**: integer weights and Heaviside threshold neurons are the native primitive. Ternary fits well within Loihi's 8-bit weight range. - **IBM TrueNorth**: configurable threshold per neurosynaptic core; ternary weights and small biases are within the supported range. - **BrainChip Akida**: edge-oriented integer-weight inference; ternary weights fit cleanly. --- ## LLM integration Threshold circuits can be embedded into transformer MLP layers to give a language model exact arithmetic. Standard LLMs fail at arithmetic because they interpolate over the training distribution rather than compute, so a 360M-parameter model trained on web text has seen `127 + 128 = 255` few times if at all and guesses based on pattern matching. The integration freezes the circuits and trains only the interface layers that: 1. Extract operands from token embeddings. 2. Route computation through the appropriate circuit. 3. Inject the result back into the residual stream. The model learns *call dispatch*; the arithmetic is already solved. ### Architecture ``` x ──┬── MLP path ─────────────────┬── + ── output │ │ └── BitExtractor ── Circuit ──┴── BitInjector │ Router (learned weighting) ``` Augmented MLP forward pass: ```python def forward(x): # x: [batch, seq, d_model=960] mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x)) a_bits, b_bits = self.bit_extractor(x) # [batch, seq, 8] each result_bits, carry = self.circuits.add_8bit(a_bits, b_bits) flags = self.compute_flags(result_bits, carry) circuit_delta = self.bit_injector(result_bits, flags) route_weights = self.router(x) # [batch, seq, 2] softmax return mlp_out + route_weights[..., 1:2] * circuit_delta ``` ### Target model The reference integration uses HuggingFace's [SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct). See [`llm_integration/SMOLLM2_ARCHITECTURE.md`](llm_integration/smollm2/SMOLLM2_ARCHITECTURE.md) for the full technical analysis. | Property | Value | |----------|-------| | Parameters | 361.82 M | | Hidden dimension | 960 (matches the extractor input) | | Layers | 32 transformer blocks | | Attention | 15 query heads, 5 KV heads (GQA) | | MLP | SwiGLU (960 → 2560 → 960) | | Position encoding | RoPE (theta = 100k, max 8192) | Digits tokenize individually (`"47 + 86"` → `['4', '7', ' +', ' ', '8', '6']`, with digit token IDs `32 + digit_value`), which makes position-based operand extraction practical. ### Gradient flow Heaviside has zero gradient almost everywhere. The implementation uses a straight-through estimator: ```python class HeavisideSTE(torch.autograd.Function): @staticmethod def forward(ctx, x): return (x >= 0).float() @staticmethod def backward(ctx, grad_output): return grad_output ``` At inference, Heaviside is the true step function; if the extractor identifies operands correctly, the circuit produces the correct result by construction. ### Baseline SmolLM2-360M-Instruct on randomized 8-bit arithmetic (2,000 cases, operands uniform on [0, 255], generous answer extraction): | Operation | Accuracy | |-----------|----------| | Addition | 35.92% | | Subtraction | 17.72% | | Multiplication | 1.25% | | Greater than | 14.37% | | Less than | 4.31% | | Equality | 0.28% | | **Overall** | **11.90%** (238/2000) | Multiplication accuracy at 1.25% is essentially random over the output space. Comparison operations often echo the expression rather than evaluate it. Even addition fails roughly two-thirds of the time on full 8-bit operands. Performance degrades further as operand magnitude increases: edge cases like `127 + 128` are almost never correct. The frozen threshold circuits reach 100% on the same task when given correctly formatted bit inputs (10,000 random cases, every operation). The integration challenge is therefore the extractor, not the arithmetic. ### Trainable parameters (SmolLM2, hidden_dim = 960) | Component | Parameters | Description | |-----------|------------|-------------| | AttentionPooling | ~3.7 M | 4-head attention over the sequence | | MultiHeadBitExtractor (× 2) | ~245 K each | 8 per-bit MLPs for A and B | | OpRouter | ~246 K | 960 → 256 → 6 MLP | | **Extractor total** | **~4.4 M** | full extraction module | Alternative architectures: `PositionExtractor` (~1.5 M, position-specific, no attention), `DigitExtractor` (~1.2 M, predicts digits 0–9 instead of bits), `HybridExtractor` (digit lookup with MLP fallback for word numerals). With `--unfreeze_layers 4` an additional ~39.3 M trainable parameters open up in the top four transformer layers. ### Training ```bash python train.py --mode router --epochs 100 # sanity check python train.py --mode llm --epochs 100 --batch_size 256 # frozen LLM python train.py --mode llm --unfreeze_layers 4 --batch_size 4096 # fine-tune top layers ``` Loss components: BCE on output bits, BCE on extracted A and B bits (2× weight), and CE on operation classification. Curriculum runs 0–9 → 0–99 → 0–255. Optimizer is AdamW, lr 3e-4, gradient clipping 1.0. --- ## Repository layout ``` neural_computer.safetensors canonical model (32-bit, 64 KB, ~8.61M params) variants/ 18 fitness variants + 9 standalone machines (neural_subleq8, neural_rv32, neural_matrix8, neural_subleq8io, neural_reflect, neural_attractor, neural_reversible, neural_ca, neural_tile) src/ the library (run scripts as `python src/.py`) ├── build.py generator (one safetensors per invocation; also `subleq`, `rv32`) ├── quantize.py min integer dtypes + ternary verification/repair ├── eval.py gate-level fitness suite, NetlistEvaluator, float oracles, reference CPU ├── eval_all.py variant-agnostic harness + manifest-sized threshold CPU ├── machines.py neural_subleq8 + neural_rv32 runtimes, references, assemblers, │ RV32 object loader, and the test suites (subleq / rv32 / rv32-c) ├── matrix8.py compiles the CPU to a recurrent ternary matrix stack; the │ exhaustive equality suite and the analog crossbar simulation ├── constructor8.py the neural_subleq8io host, the recipe codec, and the universal │ constructor / self-reproduction suite ├── reflect.py neural_reflect: a universal threshold interpreter whose state │ holds its own gate table; universality / quine / matrix suites ├── attractor.py neural_attractor: energy-based solver (forward eval, inversion, │ SAT) built from a QUBO gate-gadget compiler ├── reversible.py neural_reversible: reversible threshold gates, Cuccaro ALU, │ Bennett construction ├── reversible_cpu.py reversible register machine: bijective step, backward execution ├── reversible_prog.py structured reversible programs (multiply, Fibonacci, Janus IF) ├── ca.py neural_ca: reversible Margolus cellular automaton, threshold │ block rule, lattice reversibility and billiard-ball dynamics └── tile.py neural_tile: abstract tile assembly, threshold binding rule, XOR/Sierpinski and binary-counter tile sets tools/ build_all.py (build + quantize + verify every profile), cpu_programs.py (assembler + CPU program suite), test_cpu.py (program suite vs a variant), play.py (interactive demo), prune_weights.py (GPU-batched weight reduction), build_attractor.py / test_attractor.py (neural_attractor), build_reversible.py / reversible_matrix.py (neural_reversible), build_ca.py (neural_ca matrix tile), build_tile.py (neural_tile) llm_integration/ SmolLM2 extractor + circuit wrapper + training code ├── circuits.py FrozenThresholdCircuits (loads safetensors, exposes │ add_8bit / sub_8bit / mul_8bit / compare_*) ├── model.py Extractor variants, ArithmeticModel ├── train.py router / interface / llm training modes ├── fitness.py randomized fitness function ├── baseline.py vanilla SmolLM2 baseline measurement ├── trained/ checkpointed extractor weights └── smollm2/ ├── SMOLLM2_ARCHITECTURE.md architecture analysis ├── analyze_smollm2.py analysis script └── smollm2_analysis.json analysis output ``` --- ## Citation ```bibtex @misc{threshold-computers, title={threshold-computers: A Family of Turing-Complete Threshold-Logic Machines}, author={Norton, Charles}, year={2026}, howpublished={Hugging Face}, url={https://huggingface.co/phanerozoic/threshold-computers} } ``` --- ## License MIT --- ## References 1. McCulloch & Pitts (1943). *A Logical Calculus of Ideas Immanent in Nervous Activity.* 2. Muroga (1971). *Threshold Logic and Its Applications.* 3. Siegelmann & Sontag (1995). *On the Computational Power of Neural Nets.* 4. Bengio et al. (2013). *Estimating or Propagating Gradients Through Stochastic Neurons.* 5. Ma et al. (2024). *The Era of 1-bit LLMs* (BitNet b1.58). 6. HuggingFace (2024). *SmolLM2: Small Language Models* — [model card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct). 7. Vaswani et al. (2017). *Attention Is All You Need.* 8. Su et al. (2021). *RoFormer: Enhanced Transformer with Rotary Position Embedding.*