README.md

---
license: mit
tags:
- threshold-logic
- neuromorphic
- computer-architecture
- turing-complete
- recurrent-neural-network
- in-memory-computing
- self-replication
- loihi
- truenorth
- akida
---

# threshold-computers

A family of Turing-complete machines implemented entirely as threshold logic gates. Every gate, from Boolean primitives to arithmetic to control flow, is a single threshold neuron of the form:

```
output = 1 if (Σ wᵢ·xᵢ + b) ≥ 0 else 0
```

**Every weight in the file is in {-1, 0, 1}.** Biases are integers. Activations are the Heaviside step. Nothing else. The library was originally built with positional weights up to ±2³¹ for wide single-layer comparators; those have all been replaced with bit-cascaded multi-layer equivalents that use only ternary weights and small integer biases. Threshold-gate evaluation reduces to a popcount minus a popcount plus a bias, which is exactly what neuromorphic chips and FPGAs natively support.

The repository ships eighteen prebuilt configurations spanning three data-path widths (8, 16, 32 bits) and six memory sizes (0 B to 64 KB). The canonical file at the repo root is the largest of these: a 32-bit data path with a 64 KB address space and ~8.61 M tensor elements (~8.46 M gate weights and biases; the rest is `.inputs` routing metadata).

```
neural_computer.safetensors        32-bit data, 64 KB memory, ~8.61M params (canonical)
variants/neural_computer{8,16,32}.safetensors                    full memory (64 KB)
variants/neural_computer{8,16,32}_reduced.safetensors            4 KB memory
variants/neural_computer{8,16,32}_small.safetensors              1 KB memory
variants/neural_computer{8,16,32}_scratchpad.safetensors         256 B memory
variants/neural_computer{8,16,32}_registers.safetensors          16 B memory
variants/neural_alu{8,16,32}.safetensors                         pure ALU, no memory
variants/neural_subleq8.safetensors                              one-instruction machine (SUBLEQ)
variants/neural_rv32.safetensors                                 RV32IM + F-subset RISC-V processor
variants/neural_matrix8.safetensors                              the CPU as one recurrent ternary matrix stack
variants/neural_subleq8io.safetensors                            SUBLEQ host for the universal constructor
variants/neural_reflect.safetensors                              interpreter whose state holds its own weights
variants/neural_attractor.safetensors                            energy-based solver; a multiplier run backward factors
variants/neural_reversible.safetensors                           reversible arithmetic core, a bijection with no erasure
variants/neural_ca.safetensors                                   reversible cellular-automaton medium (no processor)
variants/neural_tile.safetensors                                 self-assembling tile computer (computation as growth)
```

Nine further machines are detailed in their own sections below, and together
they carry the family from the smallest possible processor to several results
about what a threshold network can be. `neural_subleq8` is a Turing-complete
one-instruction computer whose entire control flow is a single threshold
neuron. `neural_rv32` is a RISC-V processor (RV32IM plus an F subset) that
runs stock-compiler C, with dual-issue execution, memory-mapped I/O, and a
self-referential NEUR opcode. `neural_matrix8` drops the gate graph entirely:
the whole processor is compiled into a fixed stack of ternary weight matrices
with a Heaviside step between them, so one clock cycle is one matrix-vector
product followed by thresholding, exactly what a resistive or photonic
crossbar computes. `neural_subleq8io` hosts a universal constructor: a
program that reads a description of any machine in the family and prints that
machine's weight file byte for byte, self-reproduction being the case where
the description is its own. And `neural_reflect` is a universal interpreter
over ternary threshold netlists: its fixed transition circuit evaluates a
netlist held in the writable state, so the machine it runs is stored data that
a program reads, edits, and reproduces. `neural_attractor` drops sequential
control entirely: a circuit compiles to an energy function whose minimum is its
consistent assignment, so clamping different wires runs the same network forward
to evaluate, backward to invert (a multiplier run backward returns factors), or
as a SAT solver. And `neural_reversible` makes the entire state transition a
bijection, so no step erases information and the machine runs backward to
reconstruct its input, a processor with no Landauer erasure floor. And
`neural_ca` has no processor at all: one fixed reversible rule applied to every
2x2 block of a lattice (a Margolus cellular automaton), where a particle
collision computes an AND gate and ballistic transport plus collisions are the
billiard-ball universality primitives. And `neural_tile` computes by
self-assembly: a tile set whose binding rule is a threshold gate grows a crystal
that is the trace of a computation, a Sierpinski (Rule 90) pattern or a binary
counter, Turing-universal at temperature 2 (Winfree 1998).

---

## Quick start

```python
import torch
from safetensors.torch import load_file

tensors = load_file("neural_computer.safetensors")

def heaviside(x):
    return (x >= 0).float()

# AND gate: fires when both inputs are 1
w = tensors['boolean.and.weight']  # [2]
b = tensors['boolean.and.bias']    # [1]
for a, c in [(0, 0), (0, 1), (1, 0), (1, 1)]:
    out = heaviside((torch.tensor([a, c], dtype=torch.float32) * w).sum() + b)
    print(f"AND({a}, {c}) = {int(out.item())}")
```

Run the full circuit verification suite against any variant:

```bash
python src/eval_all.py variants/                              # 18 fitness variants (machines skipped)
python src/eval_all.py neural_computer.safetensors            # the canonical file
python src/eval_all.py --cpu-program variants/                # also run an assembled
                                                          # program through the
                                                          # threshold-gated CPU
```

`src/eval_all.py` reads each variant's manifest, runs a gate-level fitness suite (13,900–15,900 tests per variant covering Boolean, arithmetic, ALU, control, modular, error-detection, threshold, and float circuits, including end-to-end evaluation of the composed float pipelines from the shipped wiring metadata — see the Verification table), and optionally executes a small assembled program through a manifest-sized threshold CPU plus a chained 16- or 32-bit ALU sequence on wider variants.

For an interactive walkthrough that exercises Boolean gates, the 8-bit ALU, mod-5 divisibility, and a CPU loop end-to-end:

```bash
python tools/play.py                                      # 1 KB demo, runs in seconds
python tools/play.py --model neural_computer.safetensors  # 64 KB, slower
```

For end-to-end CPU validation (Fibonacci, sum 1..N, bubble sort, self-modifying JMP, all eight conditional jumps, CALL stack semantics, MUL cross-checked against repeated ADD, DIV cross-checked against repeated SUB, a bitwise AND/OR/XOR/SHL/SHR chain, and the architectural flag policy):

```bash
python tools/test_cpu.py                                     # default: 1 KB, ~2 s
python tools/test_cpu.py --model neural_computer.safetensors # 64 KB canonical, ~2 min
python tools/test_cpu.py --only fib,sum_n                    # subset of suite
```

Each program is assembled by a small Python assembler (`tools/cpu_programs.py`) and run through the threshold-gated CPU; the driver verifies expected memory contents at HALT.

---

## Execution model

A self-contained machine. State goes in, state comes out:

- **Pure tensor computation**: state in, state out
- **Frozen circuits**: integer weights, Heaviside activation
- **ACT execution**: internal loop until `HALT`
- **No external orchestration**: one forward pass equals one complete program execution

Every datapath operation runs through threshold gates: ALU arithmetic
(ADD/SUB via ripple-carry, MUL via partial-product AND gates and shift-add,
DIV via per-stage bit-cascade comparators), bitwise logic, shifts,
comparisons, conditional-jump PC muxing, stack-pointer stepping, and all
memory reads and writes. Instruction decode (a 4-to-16 opcode one-hot
decoder) and PC sequencing (the PC+2/PC+4 increment chains and the next-PC
mux) are also threshold gates now; register-field indexing and the bit
complements feeding negated-condition selects remain structural routing.

```
            ┌─────────────────────────────┐
            │      Initial State          │
            │  [PC|Regs|Flags|Memory...]  │
            └─────────────┬───────────────┘
                          ▼
            ┌─────────────────────────────┐
            │   Threshold Circuit Layer   │
            │  ┌───────────────────────┐  │
            │  │   Fetch: PC → Instr   │  │
            │  ├───────────────────────┤  │
            │  │   Decode: Opcode/Ops  │  │
            │  ├───────────────────────┤  │
            │  │   Execute: ALU/Mem    │  │
            │  ├───────────────────────┤  │
            │  │   Writeback: Results  │  │
            │  ├───────────────────────┤  │
            │  │   PC Update           │  │
            │  └───────────┬───────────┘  │
            │              │              │
            │         ┌────▼────┐         │
            │         │ HALTED? │         │
            │         └────┬────┘         │
            │       no ────┴──── yes      │
            │        │           │        │
            │        ▼           ▼        │
            │     [loop]      [exit]      │
            └─────────────┬───────────────┘
                          ▼
            ┌─────────────────────────────┐
            │       Final State           │
            └─────────────────────────────┘
```

### Instruction set

| Opcode | Mnemonic | Operation |
|--------|----------|-----------|
| 0x0 | ADD | R[d] = R[a] + R[b] |
| 0x1 | SUB | R[d] = R[a] - R[b] |
| 0x2 | AND | R[d] = R[a] & R[b] |
| 0x3 | OR | R[d] = R[a] \| R[b] |
| 0x4 | XOR | R[d] = R[a] ^ R[b] |
| 0x5 | SHL | R[d] = R[a] << 1 |
| 0x6 | SHR | R[d] = R[a] >> 1 |
| 0x7 | MUL | R[d] = R[a] * R[b] |
| 0x8 | DIV | R[d] = R[a] / R[b] |
| 0x9 | CMP | flags = R[a] - R[b] |
| 0xA | LOAD | R[d] = M[addr] |
| 0xB | STORE | M[addr] = R[s] |
| 0xC | JMP | PC = addr |
| 0xD | Jcc | PC = addr if cond (imm8[2:0]: 0=Z, 1=NZ, 2=C, 3=NC, 4=N, 5=P, 6=V, 7=NV) |
| 0xE | CALL | push PC; PC = addr |
| 0xF | HALT | stop execution |

**Flag policy.** Only ADD, SUB, MUL, and CMP write the FLAGS register; AND,
OR, XOR, SHL, SHR, DIV, LOAD, STORE, and all control transfers leave it
unchanged. Branch on the arithmetic op that set the condition — intervening
bitwise or shift instructions do not disturb it. The `flags_policy` program
in the CPU suite pins this behavior.

| Flag | ADD | SUB / CMP | MUL |
|---|---|---|---|
| Z | result == 0 | result == 0 | result == 0 |
| N | result bit 7 | result bit 7 | result bit 7 |
| C | carry-out | 1 when no borrow (`a >= b`) | 0 |
| V | signed overflow | signed overflow | 0 |

### State tensor layout

The **state tensor** uses MSB-first bit ordering: index 0 of each multi-bit field is the most-significant bit. So `R0[0]` is bit 7 of the architectural register, `R0[7]` is bit 0.

```
[ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ]
```

`N` is the address width (configurable, 0–16). Flags are ordered `Z, N, C, V`. Control bits are ordered `HALT, MEM_WE, MEM_RE, RESERVED`.

#### Bit ordering, one rule per scope

The state tensor's MSB-first convention does **not** propagate to subcircuit ports. Each subcircuit names its operand bits in its own scope:

| Scope | Convention | Example |
|---|---|---|
| State tensor | MSB-first (index 0 = MSB) | `R0[0]` is bit 7 of register R0 |
| Integer subcircuit ports (`$a[i]`, `$b[i]`) | LSB-indexed (index 0 = LSB) | `$a[0]` is bit 0 of operand `a` |
| Ripple-carry full adders (`fa0..fa7`) | LSB-first (fa0 = LSB) | `fa0` consumes `$a[0]` and `$b[0]` |
| Composed float circuits (`float16.add` etc.) | MSB-first operand words | `$a[0]` is the sign bit, `$a[1..E]` the exponent |
| Float result gates (`exp_out.bit{k}`, `frac_out.bit{k}`) | LSB-first field bits | `frac_out.bit0` is the fraction LSB |
| Instruction word | MSB-first (bit 15 = opcode high) | bit 15 is `opcode[3]` |

Worked example for `arithmetic.ripplecarry8bit`:

- Inputs: `$a[0]..$a[7]` and `$b[0]..$b[7]` where `$a[0]` is the LSB of `a`. To add `a = 0x05 = 0b00000101` and `b = 0x03`, drive `a[0]=1, a[1]=0, a[2]=1` (rest 0) and `b[0]=1, b[1]=1` (rest 0).
- Outputs: `fa0.ha2.sum.layer2`..`fa7.ha2.sum.layer2` are sum bits 0..7 (LSB to MSB), and `fa7.carry_or` is the final carry-out. The 8-bit result is `{fa7..fa0}` reading high-to-low.

This is also how `safetensors2verilog`'s threshold-logic frontend exposes the ports of any extracted subcircuit; `python -m safetensors2verilog ... --inspect` prints the port contract for any extracted circuit.

### Instruction encoding (16-bit, MSB-first)

```
15..12  11..10  9..8  7..0
opcode  rd      rs    imm8
```

Interpretation:
- **R-type**: `rd = rd op rs` (imm8 ignored)
- **I-type**: `rd = op rd, imm8` (rs ignored)
- **Address-extended**: `LOAD`, `STORE`, `JMP`, `Jcc`, `CALL` consume the next word as a 16-bit address (big-endian); `imm8` is reserved and the PC skips 4 bytes when the jump is not taken.

### Circuit categories

| Category | Circuits | Examples |
|----------|----------|----------|
| Boolean | 9 | AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES |
| Arithmetic | 18+ | half/full adder, ripple-carry (8/16/32-bit), comparators (8/16/32-bit), 3-operand adder, A+B×C and (A+B)×C expressions |
| ALU | 8/16/32-bit | shifts, multiply, divide, INC/DEC, NEG, ROL/ROR, bitwise |
| Combinational | 10+ | MUX (2:1, 4:1, 8:1), DEMUX, 3-to-8 decoder, 8-to-3 encoder, barrel shifter, priority encoder |
| Control flow | 16 | JMP, conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV), CALL, RET, PUSH, POP |
| Memory | 3 | N-bit address decoder, read mux, write cells (packed) |
| Modular | 11 | divisibility by 2–12 (multi-layer for non-powers-of-2) |
| Threshold | 13 | k-of-n gates, majority, minority, exactly-k |
| Pattern | 10 | popcount, leading/trailing ones, symmetry |
| Error detection | 11 | parity (XOR tree), checksum, CRC, Hamming |
| Float (IEEE 754) | half + single | composed ADD, MUL, DIV, EQ/LT/LE/GT/GE pipelines plus unpack/pack/classify/normalize stages |

The float ADD/MUL/DIV/CMP circuits are **self-contained composed pipelines**: their complete internal wiring ships in the files as `.inputs` metadata, their external inputs are the raw operand words (`$a[0]` sign, `$a[1..E]` exponent, then mantissa; MSB-first), and the eval suite reconstructs and executes each netlist end to end from that metadata alone (`NetlistEvaluator`). Comparisons are fully IEEE (NaN unordered, `+0 == -0`, subnormal ordering, mixed signs). The arithmetic is round-to-nearest-even, **bit-exact to IEEE hardware** including the subnormal range, with exact specials (NaN, infinities including `inf - inf`/`inf * 0`/`0/0`/`inf/inf` → NaN and `x/0` → inf, signed zeros), subnormal operands, and gradual-underflow subnormal results (each oracle validated bit-exact against numpy over every class). Because the tests run from each file's own routing metadata, they prove the artifact is self-contained: `safetensors2verilog`-style extraction of `float16.add` and friends has everything it needs to emit a single composed module.

### Tensor naming

```
{category}.{circuit}[.{layer}][.{component}].{weight|bias}

Examples:
  boolean.and.weight
  boolean.xor.layer1.neuron1.weight
  arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight
  modular.mod5.eq.k15.bit3.match.weight
  error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias
```

Memory circuits are stored as packed tensors so the safetensors header stays manageable (`memory.addr_decode.weight`, `memory.read.and.weight`, `memory.write.and_old.weight`, etc.).

---

## Bit widths and memory profiles

The build tool emits one of 51 functionally distinct configurations: three data-path widths × seventeen address widths (0–16, where 0 means no memory).

**Bit widths** (`--bits`):

| Width | Range | Use case |
|-------|-------|----------|
| 8 | 0–255 | full CPU, legacy compatibility |
| 16 | 0–65,535 | extended arithmetic |
| 32 | 0–4,294,967,295 | practical arithmetic ranges |

**Memory profiles** (`-m`):

| Profile | Size | Addr bits | Filename suffix |
|---------|------|-----------|-----------------|
| `none` | 0 B | 0 | (uses `alu` instead of `computer`) |
| `registers` | 16 B | 4 | `_registers` |
| `scratchpad` | 256 B | 8 | `_scratchpad` |
| `small` | 1 KB | 10 | `_small` |
| `reduced` | 4 KB | 12 | `_reduced` |
| `full` | 64 KB | 16 | (none) |

Auto-generated filename: `neural_{alu|computer}{BITS}[_{MEMORY}].safetensors`. Custom address widths via `-a N` produce `_addrN`.

```bash
python src/build.py --bits 32 --apply all              # neural_computer32.safetensors
python src/build.py --bits 8 -m none --apply all       # neural_alu8.safetensors
python src/build.py --bits 16 -m small --apply all     # neural_computer16_small.safetensors
python src/build.py --bits 32 -a 6 --apply all         # neural_computer32_addr6.safetensors
```

To regenerate every named variant in one pass:

```bash
python tools/build_all.py
```

This populates `variants/` with all 18 builds, quantizes each one to the smallest signed integer dtype that exactly represents its weights (~4× reduction in tensor data, with file size dominated by the safetensors header on the smaller profiles), verifies the strictly ternary weight invariant (`--ternary --strict`, so a build with any non-ternary weight fails loudly), stamps the `weight_quantization` metadata field, and runs `src/eval.py` on each as a sanity check.

The quantizer is also available standalone:

```bash
python src/quantize.py path/to/file.safetensors           # in-place
python src/quantize.py variants/                          # whole directory
python src/quantize.py model.safetensors -o quantized.safetensors
python src/quantize.py file.safetensors --ternary         # push toward {-1, 0, 1} weights
python src/quantize.py file.safetensors --ternary --strict  # error if any weight is non-ternary
```

Every weight and bias tensor in the canonical model fits in `int8`. The eval pipeline promotes weights to `float32` on load, so integer storage is exact and transparent.

**Ternary mode.** `src/build.py` emits only ternary weights: identity buffers are `weight=1, bias=-1` (`H(x - 1)`), and the comparators, modular detectors, and division stages that previously required positional weights up to ±2³¹ are bit-cascaded multi-layer equivalents. With `--ternary`, the quantizer verifies this and repairs legacy files: it rewrites historical single-input `weight=±2` buffers as `weight=±1` with the bias adjusted to preserve the heaviside output for binary inputs (`H(2x - 1) ≡ H(x - 1)`), and rebuilds pre-bit-cascade modular detectors (moduli already in bit-cascade form are left untouched, routing metadata included). `--strict` fails if any weight tensor remains non-ternary. Every shipped file carries the metadata field `weight_quantization: ternary`; a repaired file with remaining violations would be stamped `ternary_partial`.

---

## Verification

| Category | Coverage | Notes |
|----------|----------|-------|
| Boolean gates | exhaustive | all 2^n input combinations |
| Arithmetic (8-bit) | strategic sampling | edge values + diagonal pairs; ~50 cases per circuit |
| Arithmetic (16/32-bit) | strategic sampling | width-scaled extremes, alternating patterns, byte-boundary carries |
| ALU primitives (8/16/32-bit) | strategic sampling | edge inputs per operation; DIV comparators driven along real restoring-division traces |
| Control flow | exhaustive | all 2^3 input combinations per Jcc, per address bit |
| Threshold k-of-n | exhaustive | all 256 8-bit popcounts |
| Modular (all moduli, 8-bit input) | exhaustive | every value in [0, 255] |
| Parity | exhaustive | every value in [0, 255] |
| Pattern recognition | exhaustive | every value in [0, 255] |
| Combinational logic | exhaustive | full input space per gate |
| Float unpack/pack | exhaustive, functional | every bit gate driven with 0 and 1 (identity) |
| Float classify | functional | IEEE 754 categories (zero, subnormal, normal, inf, NaN) at edge encodings, both widths |
| Float CMP (composed) | functional, exact IEEE | full netlist rebuilt from the shipped `.inputs` metadata and evaluated end to end; NaN unordered, signed zeros, subnormal ordering, mixed signs — all five predicates, both widths |
| Float ADD/MUL/DIV (composed) | functional, bit-exact to IEEE hardware | same metadata-driven evaluation: exact specials, subnormal operands and gradual-underflow results, round-to-nearest-even; cross-checked against native float arithmetic |
| Memory / manifest | structure checks | packed-tensor shapes against the manifest |
| CPU integration | program-level | ten assembled programs (Fibonacci, sum, sort, self-modifying JMP, all eight Jcc, CALL stack push, MUL vs repeated ADD, DIV vs repeated SUB, a bitwise AND/OR/XOR/SHL/SHR pipeline, and the flag-policy pin) |

The 8-bit arithmetic and ALU tests use strategic sampling rather than the full 65,536-case sweep because exhaustive coverage at 8-bit is feasible but not necessary given that the circuits are constructed gate-by-gate. The 16-bit and 32-bit arithmetic tests sample edge cases only; full exhaustive coverage at those widths is infeasible without specialized hardware.

`src/eval_all.py` runs the unified suite. Exit code is the number of failing variants (0 means all pass). **Testing is evaluation, not rebuilding**: `python src/eval_all.py variants/` scores all 18 fitness variants against the shipped weights in about two minutes (~6 s each, the composed float netlists evaluated in `NetlistEvaluator`'s leveled mode) and cleanly skips the nine standalone machines. Rebuilding the models (`tools/build_all.py`, ~50 min for all 18) is a separate step, needed only when the circuit constructions in `src/build.py` change; routine verification never rebuilds. The batched evaluator is population-safe: every chained intermediate (carry, borrow, mux select) is computed per population slot, so `tools/prune_weights.py`'s parallel fitness screens are exact rather than slot-0 approximations.

---

## neural_subleq8 — the one-instruction machine

The minimal member of the family: a Turing-complete computer with **no instruction decoder** (there is only one instruction), **no registers**, and **no flags** — 194 logic gates totalling 548 ternary parameters. Its one architectural decision, branch or fall through, is a single threshold neuron (`subleq.leq`) over the sign and zero of the subtraction result.

- 8-bit data, 8-bit addresses, 256 bytes of packed threshold memory, a program counter and nothing else.
- An instruction is three bytes `A B C`. Step: `M[B] = (M[B] - M[A]) mod 256`; if the result is `<= 0` in two's complement (zero or bit 7 set), `PC = C`, else `PC += 3`. `PC = 0xFF` halts.
- Circuits (all wiring shipped as `.inputs` metadata): an 8-bit two's-complement subtractor, the branch neuron, a `PC + 3` incrementer, a branch mux, and the packed 256-row memory.

```bash
python src/build.py --apply subleq          # -> variants/neural_subleq8.safetensors
python src/machines.py subleq               # exhaustive datapath + lockstep programs
```

`src/machines.py subleq` evaluates the datapath **from the shipped wiring metadata** over all 65,536 operand pairs (result and branch decision both exhaustive), checks the PC mux, and runs a program suite (clear, negate-copy, add-by-double-negation, countdown loop) in full-state lockstep against a reference emulator. Third-party SUBLEQ toolchains target this machine directly once the `0xFF` halt convention is mapped.

---

## neural_rv32 — a RISC-V processor as a threshold network

The most capable member: a RISC-V CPU whose entire datapath is ternary threshold gates (`variants/neural_rv32.safetensors`, ~8.6 M parameters, 64 KB memory, strictly ternary).

- **RV32I base**: LUI, AUIPC, JAL, JALR, all six branches, LB/LH/LW/LBU/LHU, SB/SH/SW, and the full OP-IMM/OP groups. 32 × 32-bit registers (x0 zero), little-endian memory through the packed threshold circuits. ECALL halts.
- **M extension**: MUL, MULH, MULHSU, MULHU (full 64-bit product through a shift-add array with gate-level sign correction), DIV, DIVU, REM, REMU (32 restoring stages, spec-exact divide-by-zero and overflow).
- **F subset**: FLW, FSW, FMV.W.X, FMV.X.W, FADD.S, FSUB.S, FMUL.S, FDIV.S, FMADD.S, FMSUB.S, FNMADD.S, FNMSUB.S, FEQ.S, FLT.S, FLE.S, FSGNJ[N/X].S, FCVT.W.S (honors the instruction's rounding-mode field), FCVT.S.W — the arithmetic executed by the composed float32 pipelines (round-to-nearest-even, bit-exact to hardware; specials and subnormals as above; the fused multiply-add rounds `a*b+c` with a single rounding), the int/float conversions gate-routed through the priority encoder and barrel shifter and cross-checked bit-exact against native conversion.
- **NEUR (custom-0)**: `neur rd, rs1, rs2` evaluates one threshold neuron — `rd = H(popcount(rs1[7:0] & rs2[7:0]) - popcount(rs1[7:0] & rs2[15:8]) + sext(rs2[20:16]))`. Networks of NEUR instructions are neural networks running as software on the neural network; the test suite computes XOR with a two-layer NEUR net.
- **Dual issue**: two adjacent OP/OP-IMM/LUI/AUIPC instructions retire in one cycle when the gate-level hazard comparators (`rv32.hazard.*`) clear RAW and WAW dependences.
- **MMIO**: stores to `0xFF00` append a character to the console.

Signed comparisons ride the unsigned bit-cascade with sign bits complemented through NOT gates; SLL uses the barrel shifter, SRL is bit-reversal wiring over it, SRA a gate mux over the complement form. Instruction decode (opcode-class detectors and the I/S/B/U/J immediate mux) and PC sequencing (the next-PC mux over PC+4, PC+imm, and (rs1+imm)&~1) are threshold gates; register-field indexing remains fixed wiring.

```bash
python src/build.py --apply rv32                            # build the file
python src/quantize.py variants/neural_rv32.safetensors --ternary --strict
python src/machines.py rv32                                 # eight-program lockstep suite
python src/machines.py rv32-c                               # stock-compiler C, end to end
```

**Running compiled C.** `machines.py rv32-c` compiles a freestanding C program (gcd, Fibonacci, insertion sort; `rv32im`, so real `mul`/`rem`) with an unmodified clang rv32im toolchain, loads the relocatable object with an in-repo loader (no external linker — it resolves the R_RISCV relocations of one translation unit and lays the sections out flat), executes it on the threshold CPU, and checks the return value against the value computed natively. The program retires in ~300 instructions and matches exactly. Stock `rv32im` toolchains (gcc, clang, rustc) emit this ISA.

The composed circuits evaluate in a leveled mode (`NetlistEvaluator`, one padded tensor op per topological level instead of one Python step per gate), ~18× faster on the FPU-scale netlists.

---

## neural_matrix8 — the processor as one recurrent threshold network

Every machine above is a graph of individually-named gates. `neural_matrix8`
is the same processor with the graph dissolved: the verified gate wiring of
the registers-profile CPU (8-bit data, 4-bit addresses, 16 B of memory) is
compiled into a fixed sequence of ternary weight matrices `W1..Wk` with a
Heaviside step between them, acting on a single state vector that holds the
program counter, all four registers, the flags, the stack pointer, the halt
bit, and every memory bit:

```
[ PC[4] | R0[8] R1[8] R2[8] R3[8] | Z N C V | SP[4] | HALT | MEM[16][8] ]   = 173 bits
```

One instruction is one pass through the stack, `s' = H(Wk · … H(W1·s + b1) …
+ bk)`; the machine is that stack iterated until the halt bit sets, so the
whole processor is a single recurrent linear-threshold network. The
transition is total: a halted state is a fixed point (every architectural
write is gated by NOT halt), so iterating past HALT is harmless.

- **Ternary and small-integer, end to end.** The step circuit is assembled
  from the family's verified cells (two-layer OR/NAND XOR, ripple-carry full
  adders, bit-cascade comparators, restoring-division stages, 2:1 muxes,
  one-gate address-decode rows), then levelized with identity pass-throughs
  carrying live signals forward. It compiles to 108 matrices whose every
  entry is in `{-1, 0, 1}` with small integer biases (~9.1 MB on disk).
- **Equality is machine-checked bit for bit,** not sampled: all 65,536 `(a,
  b)` operand pairs for each of the ten ALU opcodes (ADD SUB AND OR XOR SHL
  SHR MUL DIV CMP) batched through the matrices against an integer reference;
  the full Jcc × flag, JMP, CALL, and LOAD/STORE control space; 2,000 uniformly
  random full states plus 500 halted-state fixed points; and per-instruction
  full-state lockstep against the gate-graph `GenericThresholdCPU` walking the
  shipped `neural_computer8_registers` weights.
- **Analog realization.** Pre-activations are integers; a firing gate sits at
  `>= 0` and a non-firing gate at `<= -1`, so a comparator threshold at `-0.5`
  gives every gate a guaranteed noise margin of exactly 0.5 (measured minimum
  over random states: 0.500). Any total analog error below 0.5 per
  pre-activation therefore reproduces the digital circuit bit for bit. Each
  matrix maps to a crossbar with conductances drawn from `{-1, 0, +1}` and one
  clock cycle is one analog matrix-vector product followed by thresholding;
  the suite injects Gaussian read noise (bit-exact through σ ≈ 0.10, breaking
  down near σ = 0.15 exactly where the error model predicts) and static
  conductance mismatch (bit-exact through σ_G = 0.10).

```bash
python src/matrix8.py build          # compile + save variants/neural_matrix8.safetensors
python src/matrix8.py verify         # exhaustive equality vs the gate graph and integer reference
python src/matrix8.py analog         # margin measurement + read-noise + conductance-mismatch sweeps
```

Because the transition is a fixed matrix stack proven equal to the gate-graph
processor, the ISA semantics carry over unchanged, and the digital circuit and
its analog crossbar realization coincide within the device's quantization and
noise margins. The processor is no longer *described by* a neural network; it
*is* one, iterated.

---

## neural_attractor — computation as energy relaxation

An energy-based threshold network with no program counter, clock, or instruction
stream. A Boolean circuit compiles to a quadratic energy
`E(s) = sum_i L[i] s_i + sum_{i<j} Q[i,j] s_i s_j` over binary wire variables such
that, with any chosen subset of wires clamped, the circuit's consistent
assignment is the global minimum. The coupling matrix `Q` and linear terms `L`
are the program; execution is relaxation toward the minimum. The relaxation step
is a threshold neuron over the same integer weights,
`s_i <- H(-(L[i] + sum_j Q[i,j] s_j))`.

Each gate adds a gadget that is non-negative and zero exactly on its truth table:
`AND` `3z + xy - 2xz - 2yz`, `OR` `x + y + z + xy - 2xz - 2yz`, `NOT`
`1 - x - z + 2xz`. These are functionally complete, so any circuit compiles and
its evaluation is the ground state by construction. The choice of clamped wires
selects the mode:

- **Forward evaluation.** Clamp the inputs; propagating the gate relations in
  topological order reaches the energy-0 fixed point. Checked bit-exact against
  integer references for ripple-carry adders and array multipliers.
- **Inversion.** Clamp the outputs and relax over the free wires. The shipped
  `neural_attractor` compiles an 8x8 array multiplier to 913 wires; it multiplies
  forward bit-exactly, and clamping the product recovers factors
  (35 = 5 x 7, 143 = 11 x 13).
- **Satisfiability.** Clamp a CNF's output wire to 1; a ground state is a
  satisfying assignment.

Forward evaluation is exact and linear in gate count. Inversion and
open-constraint solving are ground-state search over the free wires, annealed.
The target energy is 0 by construction, so a zero-energy state certifies a
correct assignment, but reaching it is NP-hard in general and not guaranteed
within a fixed annealing schedule.

```bash
python tools/build_attractor.py      # compile a multiplier to variants/neural_attractor.safetensors
python tools/test_attractor.py       # forward eval, whole-network relaxation, factoring, SAT
```

---

## neural_subleq8io and the universal constructor — a machine that prints itself

`neural_subleq8io` is the one-instruction machine extended with three
memory-mapped device cells (a tape-input byte, an input strobe, and an output
byte). The device logic lives in the runtime, exactly as `neural_rv32`'s
console does; the processor's gates are unchanged, 194 of them. On this host
runs the **universal constructor**: a 21-instruction SUBLEQ program that reads
a *construction recipe* off the tape and emits a safetensors file one byte at
a time.

`describe(file_bytes)` compiles any file in the family into a recipe; running
the constructor on that recipe emits the file back byte for byte. The
constructor is therefore universal, and the description selects what is built.
Self-reproduction is the single case where the description handed to the
constructor is that of the running machine: the program's output stream is
then the byte-for-byte safetensors file of the host itself.

The equality is machine-checked rather than observed on one run:

- the recipe codec round-trips every file in the family (all 28 shipped
  `.safetensors`, 971 MB, byte-identical and sha-verified);
- the constructor program is executed on three independently-verified
  backends — a pure-integer reference, the gate-graph `SubleqThresholdCPU`
  walking the shipped netlist, and the host itself compiled to recurrent
  ternary matrices by `matrix8`'s compiler — and they agree cycle for cycle;
- **full self-reproduction runs entirely on the threshold matrices:** the host
  compiles to 34 ternary layers, and generation 1 emits all 95,306 bytes of
  its own weight file in 359,743 recurrences, each output byte asserted
  against the target as it streams, ending on an sha match and an instruction
  count identical to the reference;
- the offspring file then boots as generation 2 and reconstructs itself,
  closing the loop across two generations.

```bash
python src/constructor8.py build     # emit variants/neural_subleq8io.safetensors
python src/constructor8.py verify    # host soundness + codec round-trip + constructor on all 3 backends
python src/constructor8.py self      # full self-reproduction on the threshold matrices, then gen-2 boot
```

A program running on the processor emits the exact tensor file that defines
that processor, and builds any other machine in the family from a description
of it: von Neumann's universal constructor, realized as ternary threshold
weights and checked to the byte.

---

## neural_reflect — an interpreter whose state holds its own weights

Every machine above has a fixed weight tensor: the weights define the machine,
the state is what it operates on, and the two are disjoint. `neural_reflect` is
a universal interpreter that holds the netlist it evaluates in its own writable
state. The fixed part is a ternary threshold circuit U (12,993 gates, 50
ternary matrices); everything machine-specific is stored data that the running
program can read, edit, copy, and replace.

The state is a flat array of 1,024 signals: a pointer register, memory-mapped
device signals (output stream, pointer advance, bank select, halt) of the kind
`neural_rv32` and `neural_subleq8io` already use, working signals, and two
netlist banks. Each stored gate record holds two input slots (address, ternary
weight, index flag), a signed bias, and an output (address, index flag). One
recurrence evaluates gate `gp` of the active bank:

```
eff(addr, idx) = (addr + (idx ? PTR : 0)) mod S
out = H( w0·sig[eff(a0,i0)] + w1·sig[eff(a1,i1)] + bias ) ;   sig[eff(oa,oidx)] = out
```

`G` recurrences sweep the netlist once; `G` sweeps settle any acyclic netlist
regardless of gate order, so U reproduces the family's combinational evaluation.

- **Universal over the family, at scale.** `compile_to_reflect` maps any
  ≤2-input family `Net` to a stored netlist; a ripple-carry adder and the
  family's own SUBLEQ datapath run through U bit-exact, the datapath checked
  against the reference over all 65,536 operand pairs and independent of the
  order the gates are stored in. Because the netlist is read by gate index
  rather than by address, the addressed signal span stays small as the gate
  count grows.
- **A complete stored machine runs inside it.** The whole SUBLEQ machine, its
  32-byte memory held in signals, executes as a 266-gate microprogram: fetch
  three operand bytes, read two data bytes, subtract, write the result back,
  branch, one instruction per sweep, with the interpreter's own indexed
  addressing serving as the memory-access hardware and the memory and program
  counter carrying across sweeps. A stored countdown program runs its loop to a
  halt with final memory matching the reference emulator, and one instruction
  is exact over 4,096 random memory states.
- **The program is its own weights.** When the netlist is address-reachable a
  gate writes into it: one program streams the netlist's own bytes out the
  output device, another computes `a & b` and then rewrites its own gate so the
  next sweep computes `a & ~b`, and a `BANK` flag runs a different resident
  machine in place. A netlist with feedback holds state across sweeps (a 2-bit
  counter), and the compiled matrix form equals the gate graph within the same
  0.5 comparator margin, measured under read noise and conductance mismatch.

```bash
python src/reflect.py verify     # universality, a hosted SUBLEQ machine, quine, metamorphosis
python src/reflect.py analog     # matrix == gate + noise / conductance-mismatch margin
```

The fixed interpreter is a substrate, universal over ternary threshold
netlists; the machine it runs is stored data that reads, rewrites, and
reproduces itself. The transition and the state it transforms are the same
tensor.

---

## neural_reversible — a processor with no logical erasure

A register machine whose entire state transition is a bijection: no step erases
information, and the same weights run backward invert it. It is built from
reversible threshold gates, CNOT (`t <- t XOR c`), Toffoli
(`t <- t XOR (a AND b)`), and Fredkin (controlled swap), each realized with the
repository's Heaviside AND/XOR gates and each a permutation of its wires by
construction, verified exhaustively.

The reversible ALU builds up from those gates. A Cuccaro ripple-carry adder
computes `b <- (a + b) mod 2^n` in place with one carry ancilla, restoring the
addend and the ancilla; subtraction is the same circuit run in reverse. XOR,
two's-complement negate, increment, rotate, and a word-level Toffoli are likewise
bijections, checked against integer references. The machine adds registers, a
program counter, a memory with reversible register/memory exchange, and a branch
register BR that makes control reversible: `PC += dir * BR` with BR = 1 for
sequential flow, and a branch toggles BR so a matched branch at the destination
restores it. The single-step transition is verified to be a bijection directly,
`step_back o step = identity` over every instruction and branch-register state;
running with dir = -1 reconstructs the input.

Irreversible functions are handled by Bennett's construction (compute into
scratch, copy the result out, uncompute the scratch), verified to map
`(a, b, c, 0, 0) -> (a, b, c, 0, f)` for `f = (a+b) XOR c` with the scratch
returned to zero, so the reversible machine computes what the irreversible
machines do without erasing.

A bijective transition erases no bits and therefore carries no Landauer floor of
`kT ln 2` per erased bit. The reversible circuits compile to the same ternary
matrix stack `neural_matrix8` uses: the 4-bit adder becomes 39 ternary matrices
with a Heaviside step, its composed transition is a verified permutation of the
state space, and every pre-activation clears the analog threshold by the same
0.5 margin, so the crossbar realization is bit-exact and information-theoretically
lossless, holding under injected read noise through sigma ~ 0.10 and static
conductance mismatch through sigma_G ~ 0.10, the same tolerances `neural_matrix8`
measures. Turning that into measured energy below the Landauer bound requires
adiabatic drive of the crossbar, which is the physical frontier; the logical
reversibility and its lossless matrix realization are proven here.

The arithmetic core ships as `variants/neural_reversible.safetensors`, an 8-bit
in-place adder (`b <- a+b`) stored as its reversible gate sequence together with
the Heaviside AND/XOR weights that realize each gate, so the file holds both the
reversible program and the threshold substrate it runs on.

```bash
python src/reversible.py       # reversible gates, Cuccaro ALU, Bennett construction
python src/reversible_cpu.py   # bijective transition and backward execution
python src/reversible_prog.py  # structured reversible programs (multiply, Fibonacci, Janus IF)
python tools/build_reversible.py   # ship + round-trip variants/neural_reversible.safetensors
python tools/reversible_matrix.py  # ternary matrix stack: permutation transition + 0.5 margin
```

---

## neural_ca — a reversible cellular automaton

A partitioned (Margolus) cellular automaton: one fixed rule applied to every 2x2
block of a lattice, with the block partition alternating each step. The state is
the lattice; there is no program counter, register file, or control circuit, and
one step updates every block.

The block rule rotates each block 180 degrees, except that two particles on a
diagonal swap to the other diagonal. It is a permutation of the sixteen block
states and its own inverse, so the lattice update is a bijection: replaying the
partition sequence in reverse reconstructs any earlier configuration (verified
over random lattices), and particle number is conserved. The rule is expressed
in the family's Heaviside threshold gates (a diagonal-pair detector XORed onto
the rotated cells) and compiles to a 6-layer ternary matrix tile that is a
permutation with a 0.5 analog margin, bit-exact under read noise through
sigma ~ 0.10 and conductance mismatch through sigma_G ~ 0.10; that tile applied
to every block is one lattice step, stored as `variants/neural_ca.safetensors`.

The dynamics are the billiard-ball model's: a single particle moves ballistically
along a diagonal at one cell per step, and two particles collide and deflect
reversibly (verified as a genuine interaction, distinct from independent motion).
A collision computes logic directly: with input particles at (2,2) and (7,7),
three output cells at step 4 carry A AND B (the deflected paths (3,6) and (6,3)),
A AND NOT B ((6,6)), and NOT A AND B ((3,3)), the billiard-ball interaction
gate, verified over all four inputs. The gates compose: launching a third
particle at (0,9) to collide with the A AND B output makes cell (3,5) carry
A AND B AND C, verified over all eight inputs, so chained collisions build larger
circuits. With mirror routing and a constant-particle source the construction is
functionally complete (Margolus 1984).

```bash
python src/ca.py             # rule bijection, lattice reversibility, ballistic motion, interaction gate
python tools/build_ca.py     # ship the block rule as a ternary matrix tile (permutation + 0.5 margin)
```

---

## neural_tile — computation as self-assembly

A tile computer in the abstract tile assembly model. The program is a finite set
of square tiles with glue labels and integer strengths on their edges; a seed is
placed and tiles accrete onto the assembly by one rule: a tile binds at an empty
site when the summed strength of its glues that match the already-present
neighbors reaches the temperature tau. That binding decision is the Heaviside
gate `H(sum_d strength_d * match_d - tau)`, weights the glue strengths and bias
`-tau`, so every attachment is a threshold neuron and the assembly grows site by
site under the family's gate. At tau = 2 the model is Turing-universal
(Winfree 1998).

Two directed tile sets are verified. The rule-tile set computes
`value(x,y) = f(value(x-1,y), value(x,y-1))` for any 2-input `f`: with `f` = XOR
the assembly is the Sierpinski triangle (Rule 90), and AND and OR grow their own
patterns, each of 529 tiles checked cell by cell against the recurrence. The
binary counter grows one integer per row: at 8-bit width it fills 255 rows and
row y encodes the integer y, with the increment carry propagating westward by
cooperative binding. Both tile sets are directed, so a unique tile binds at each
site and the assembly is deterministic.

The shipped `variants/neural_tile.safetensors` stores the counter tile set as its
glue tables together with the per-tile binding-gate weights (glue strengths) and
bias (`-tau`); reloading it regrows the counter and reproduces the binding
decision from the gate.

```bash
python src/tile.py           # binding gate, XOR/AND/OR rule tiles, binary counter
python tools/build_tile.py   # ship + regrow variants/neural_tile.safetensors
```

---

## Threshold logic

A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing the result to a threshold; the output is 1 when the sum meets or exceeds the threshold and 0 otherwise. Equivalently, it is a neuron with Heaviside step activation, integer weights, and an integer bias.

Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function, which includes AND, OR, NAND, NOR, IMPLIES, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (XOR, parity, mod-k for k not a power of two) require multiple layers.

Example gates:

```
AND: w=[1, 1], b=-2
  H(0+0-2) = 0     H(1+1-2) = 1

OR:  w=[1, 1], b=-1
  H(0+0-1) = 0     H(1+0-1) = 1

XOR: two layers (not linearly separable)
  layer 1: OR + NAND
  layer 2: AND of the two
```

A full adder is two half-adders plus a carry OR, around four threshold layers. An 8-bit ripple-carry adder is eight chained full adders, around 32 layers.

### History

Warren McCulloch and Walter Pitts introduced the threshold neuron in 1943, proving that networks of such neurons can compute any Boolean function. Their work preceded both the perceptron and modern neural networks and established the theoretical foundation for neural computation.

The 1960s saw substantial work on threshold logic synthesis. Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function can be implemented as a single threshold gate, and if so, how to compute the appropriate weights. The focus was on individual gates rather than complete systems.

Frank Rosenblatt's Mark I Perceptron (1957–1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training; the final configurations were not published. Bernard Widrow's ADALINE and MADALINE (1960–1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm.

Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing-complete. The construction relied on continuous sigmoid activations with infinite precision, not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: universality with continuous activations chosen to support gradient-based training.

### Neuromorphic hardware

Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon:

- **Intel Loihi** (2017): 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. Supports integer weights and configurable neuron dynamics.
- **IBM TrueNorth** (2014): one million neurons and 256 million synapses across a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds.
- **BrainChip Akida** (2021): edge-oriented event-based processing with integer weights.
- **SpiNNaker** (University of Manchester): ARM cores simulating spiking networks at scale.

Published work on these platforms has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1 and described that work as "a first step toward the construction of a spiking computer"; that implementation lacked instruction fetch, a program counter, memory, and control logic.

The weights in this repository implement a complete CPU: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every logic component is a threshold neuron with integer weights; opcode decode and PC sequencing run through threshold gates as well, leaving only register-field indexing as structural wiring.

---

## Hardware compatibility

All weights are in {-1, 0, 1}, all activations are Heaviside step, and every gate is a single weighted sum followed by a sign test. This eliminates multipliers entirely: each gate evaluation is a popcount of `+1`-weighted inputs minus a popcount of `-1`-weighted inputs plus an integer bias. The circuits are intended to deploy directly on:

- **FPGA**: every gate maps to a small LUT cluster (or a popcount tree of LUT4/LUT6 + carry chain). Ternary weight storage compresses to 2 bits per weight; routing collapses to bit selection.
- **Intel Loihi**: integer weights and Heaviside threshold neurons are the native primitive. Ternary fits well within Loihi's 8-bit weight range.
- **IBM TrueNorth**: configurable threshold per neurosynaptic core; ternary weights and small biases are within the supported range.
- **BrainChip Akida**: edge-oriented integer-weight inference; ternary weights fit cleanly.

---

## LLM integration

Threshold circuits can be embedded into transformer MLP layers to give a language model exact arithmetic. Standard LLMs fail at arithmetic because they interpolate over the training distribution rather than compute, so a 360M-parameter model trained on web text has seen `127 + 128 = 255` few times if at all and guesses based on pattern matching.

The integration freezes the circuits and trains only the interface layers that:

1. Extract operands from token embeddings.
2. Route computation through the appropriate circuit.
3. Inject the result back into the residual stream.

The model learns *call dispatch*; the arithmetic is already solved.

### Architecture

```
x ──┬── MLP path ─────────────────┬── + ── output
    │                             │
    └── BitExtractor ── Circuit ──┴── BitInjector
                          │
                       Router (learned weighting)
```

Augmented MLP forward pass:

```python
def forward(x):  # x: [batch, seq, d_model=960]
    mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x))

    a_bits, b_bits = self.bit_extractor(x)              # [batch, seq, 8] each
    result_bits, carry = self.circuits.add_8bit(a_bits, b_bits)
    flags = self.compute_flags(result_bits, carry)
    circuit_delta = self.bit_injector(result_bits, flags)

    route_weights = self.router(x)                       # [batch, seq, 2] softmax
    return mlp_out + route_weights[..., 1:2] * circuit_delta
```

### Target model

The reference integration uses HuggingFace's [SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct). See [`llm_integration/SMOLLM2_ARCHITECTURE.md`](llm_integration/smollm2/SMOLLM2_ARCHITECTURE.md) for the full technical analysis.

| Property | Value |
|----------|-------|
| Parameters | 361.82 M |
| Hidden dimension | 960 (matches the extractor input) |
| Layers | 32 transformer blocks |
| Attention | 15 query heads, 5 KV heads (GQA) |
| MLP | SwiGLU (960 → 2560 → 960) |
| Position encoding | RoPE (theta = 100k, max 8192) |

Digits tokenize individually (`"47 + 86"` → `['4', '7', ' +', ' ', '8', '6']`, with digit token IDs `32 + digit_value`), which makes position-based operand extraction practical.

### Gradient flow

Heaviside has zero gradient almost everywhere. The implementation uses a straight-through estimator:

```python
class HeavisideSTE(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        return (x >= 0).float()

    @staticmethod
    def backward(ctx, grad_output):
        return grad_output
```

At inference, Heaviside is the true step function; if the extractor identifies operands correctly, the circuit produces the correct result by construction.

### Baseline

SmolLM2-360M-Instruct on randomized 8-bit arithmetic (2,000 cases, operands uniform on [0, 255], generous answer extraction):

| Operation | Accuracy |
|-----------|----------|
| Addition | 35.92% |
| Subtraction | 17.72% |
| Multiplication | 1.25% |
| Greater than | 14.37% |
| Less than | 4.31% |
| Equality | 0.28% |
| **Overall** | **11.90%** (238/2000) |

Multiplication accuracy at 1.25% is essentially random over the output space. Comparison operations often echo the expression rather than evaluate it. Even addition fails roughly two-thirds of the time on full 8-bit operands. Performance degrades further as operand magnitude increases: edge cases like `127 + 128` are almost never correct.

The frozen threshold circuits reach 100% on the same task when given correctly formatted bit inputs (10,000 random cases, every operation). The integration challenge is therefore the extractor, not the arithmetic.

### Trainable parameters (SmolLM2, hidden_dim = 960)

| Component | Parameters | Description |
|-----------|------------|-------------|
| AttentionPooling | ~3.7 M | 4-head attention over the sequence |
| MultiHeadBitExtractor (× 2) | ~245 K each | 8 per-bit MLPs for A and B |
| OpRouter | ~246 K | 960 → 256 → 6 MLP |
| **Extractor total** | **~4.4 M** | full extraction module |

Alternative architectures: `PositionExtractor` (~1.5 M, position-specific, no attention), `DigitExtractor` (~1.2 M, predicts digits 0–9 instead of bits), `HybridExtractor` (digit lookup with MLP fallback for word numerals). With `--unfreeze_layers 4` an additional ~39.3 M trainable parameters open up in the top four transformer layers.

### Training

```bash
python train.py --mode router --epochs 100                          # sanity check
python train.py --mode llm --epochs 100 --batch_size 256            # frozen LLM
python train.py --mode llm --unfreeze_layers 4 --batch_size 4096    # fine-tune top layers
```

Loss components: BCE on output bits, BCE on extracted A and B bits (2× weight), and CE on operation classification. Curriculum runs 0–9 → 0–99 → 0–255. Optimizer is AdamW, lr 3e-4, gradient clipping 1.0.

---

## Repository layout

```
neural_computer.safetensors         canonical model (32-bit, 64 KB, ~8.61M params)
variants/                           18 fitness variants + 9 standalone machines
                                    (neural_subleq8, neural_rv32, neural_matrix8,
                                     neural_subleq8io, neural_reflect,
                                     neural_attractor, neural_reversible, neural_ca,
                                     neural_tile)
src/                                the library (run scripts as `python src/<name>.py`)
  ├── build.py                      generator (one safetensors per invocation; also `subleq`, `rv32`)
  ├── quantize.py                   min integer dtypes + ternary verification/repair
  ├── eval.py                       gate-level fitness suite, NetlistEvaluator, float oracles, reference CPU
  ├── eval_all.py                   variant-agnostic harness + manifest-sized threshold CPU
  ├── machines.py                   neural_subleq8 + neural_rv32 runtimes, references, assemblers,
  │                                 RV32 object loader, and the test suites (subleq / rv32 / rv32-c)
  ├── matrix8.py                    compiles the CPU to a recurrent ternary matrix stack; the
  │                                 exhaustive equality suite and the analog crossbar simulation
  ├── constructor8.py               the neural_subleq8io host, the recipe codec, and the universal
  │                                 constructor / self-reproduction suite
  ├── reflect.py                    neural_reflect: a universal threshold interpreter whose state
  │                                 holds its own gate table; universality / quine / matrix suites
  ├── attractor.py                  neural_attractor: energy-based solver (forward eval, inversion,
  │                                 SAT) built from a QUBO gate-gadget compiler
  ├── reversible.py                 neural_reversible: reversible threshold gates, Cuccaro ALU,
  │                                 Bennett construction
  ├── reversible_cpu.py             reversible register machine: bijective step, backward execution
  ├── reversible_prog.py            structured reversible programs (multiply, Fibonacci, Janus IF)
  ├── ca.py                         neural_ca: reversible Margolus cellular automaton, threshold
  │                                 block rule, lattice reversibility and billiard-ball dynamics
  └── tile.py                       neural_tile: abstract tile assembly, threshold binding rule,
                                    XOR/Sierpinski and binary-counter tile sets
tools/                              build_all.py (build + quantize + verify every profile),
                                    cpu_programs.py (assembler + CPU program suite), test_cpu.py
                                    (program suite vs a variant), play.py (interactive demo),
                                    prune_weights.py (GPU-batched weight reduction),
                                    build_attractor.py / test_attractor.py (neural_attractor),
                                    build_reversible.py / reversible_matrix.py (neural_reversible),
                                    build_ca.py (neural_ca matrix tile), build_tile.py (neural_tile)
llm_integration/                    SmolLM2 extractor + circuit wrapper + training code
  ├── circuits.py                   FrozenThresholdCircuits (loads safetensors, exposes
  │                                 add_8bit / sub_8bit / mul_8bit / compare_*)
  ├── model.py                      Extractor variants, ArithmeticModel
  ├── train.py                      router / interface / llm training modes
  ├── fitness.py                    randomized fitness function
  ├── baseline.py                   vanilla SmolLM2 baseline measurement
  ├── trained/                      checkpointed extractor weights
  └── smollm2/
      ├── SMOLLM2_ARCHITECTURE.md   architecture analysis
      ├── analyze_smollm2.py        analysis script
      └── smollm2_analysis.json     analysis output
```

---

## Citation

```bibtex
@misc{threshold-computers,
  title={threshold-computers: A Family of Turing-Complete Threshold-Logic Machines},
  author={Norton, Charles},
  year={2026},
  howpublished={Hugging Face},
  url={https://huggingface.co/phanerozoic/threshold-computers}
}
```

---

## License

MIT

---

## References

1. McCulloch & Pitts (1943). *A Logical Calculus of Ideas Immanent in Nervous Activity.*
2. Muroga (1971). *Threshold Logic and Its Applications.*
3. Siegelmann & Sontag (1995). *On the Computational Power of Neural Nets.*
4. Bengio et al. (2013). *Estimating or Propagating Gradients Through Stochastic Neurons.*
5. Ma et al. (2024). *The Era of 1-bit LLMs* (BitNet b1.58).
6. HuggingFace (2024). *SmolLM2: Small Language Models* — [model card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct).
7. Vaswani et al. (2017). *Attention Is All You Need.*
8. Su et al. (2021). *RoFormer: Enhanced Transformer with Rotary Position Embedding.*