8-bit threshold-logic CPU family: ternary-weight gate networks from a one-instruction SUBLEQ machine to an RV32IM plus F-subset RISC-V processor that runs stock-compiler C; composed IEEE-754 float pipelines with round-to-nearest-even bit-exact to hardware and metadata-driven verification; fully-wired rv32 datapath, FCVT int/float conversions, single gate-routed CPU runtime, leveled fast evaluation; single-file docs and consolidated machine runtime; strict-ternary build

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tensors.txt filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,4 @@
+__pycache__/
+*.pyc
+.pt file
+.eval_cache/

README.md

@@ -0,0 +1,613 @@
+---
+license: mit
+tags:
+- threshold-logic
+- neuromorphic
+- computer-architecture
+- turing-complete
+- loihi
+- truenorth
+- akida
+---
+
+# 8bit-threshold-computer
+
+A Turing-complete CPU implemented entirely as threshold logic gates. Every gate, from Boolean primitives to arithmetic to control flow, is a single threshold neuron of the form:
+
+```
+output = 1 if (Σ wᵢ·xᵢ + b) ≥ 0 else 0
+```
+
+**Every weight in the file is in {-1, 0, 1}.** Biases are integers. Activations are the Heaviside step. Nothing else. The library was originally built with positional weights up to ±2³¹ for wide single-layer comparators; those have all been replaced with bit-cascaded multi-layer equivalents that use only ternary weights and small integer biases. Threshold-gate evaluation reduces to a popcount minus a popcount plus a bias, which is exactly what neuromorphic chips and FPGAs natively support.
+
+The repository ships eighteen prebuilt configurations spanning three data-path widths (8, 16, 32 bits) and six memory sizes (0 B to 64 KB). The canonical file at the repo root is the largest of these: a 32-bit data path with a 64 KB address space and ~8.61 M tensor elements (~8.46 M gate weights and biases; the rest is `.inputs` routing metadata).
+
+```
+neural_computer.safetensors        32-bit data, 64 KB memory, ~8.61M params (canonical)
+variants/neural_computer{8,16,32}.safetensors                    full memory (64 KB)
+variants/neural_computer{8,16,32}_reduced.safetensors            4 KB memory
+variants/neural_computer{8,16,32}_small.safetensors              1 KB memory
+variants/neural_computer{8,16,32}_scratchpad.safetensors         256 B memory
+variants/neural_computer{8,16,32}_registers.safetensors          16 B memory
+variants/neural_alu{8,16,32}.safetensors                         pure ALU, no memory
+variants/neural_subleq8.safetensors                              one-instruction machine (SUBLEQ)
+variants/neural_rv32.safetensors                                 RV32IM + F-subset RISC-V processor
+```
+
+Two further machines mark the endpoints of the family, each detailed in
+its own section below: `neural_subleq8`, a Turing-complete one-instruction
+computer whose entire control flow is a single threshold neuron; and
+`neural_rv32`, a RISC-V processor (RV32IM + an F subset) that runs
+stock-compiler C, with dual-issue execution, memory-mapped I/O, and a
+self-referential NEUR opcode.
+
+---
+
+## Quick start
+
+```python
+import torch
+from safetensors.torch import load_file
+
+tensors = load_file("neural_computer.safetensors")
+
+def heaviside(x):
+    return (x >= 0).float()
+
+# AND gate: fires when both inputs are 1
+w = tensors['boolean.and.weight']  # [2]
+b = tensors['boolean.and.bias']    # [1]
+for a, c in [(0, 0), (0, 1), (1, 0), (1, 1)]:
+    out = heaviside((torch.tensor([a, c], dtype=torch.float32) * w).sum() + b)
+    print(f"AND({a}, {c}) = {int(out.item())}")
+```
+
+Run the full circuit verification suite against any variant:
+
+```bash
+python eval_all.py variants/                              # 18 fitness variants (machines skipped)
+python eval_all.py neural_computer.safetensors            # the canonical file
+python eval_all.py --cpu-program variants/                # also run an assembled
+                                                          # program through the
+                                                          # threshold-gated CPU
+```
+
+`eval_all.py` reads each variant's manifest, runs a gate-level fitness suite (13,900–15,900 tests per variant covering Boolean, arithmetic, ALU, control, modular, error-detection, threshold, and float circuits, including end-to-end evaluation of the composed float pipelines from the shipped wiring metadata — see the Verification table), and optionally executes a small assembled program through a manifest-sized threshold CPU plus a chained 16- or 32-bit ALU sequence on wider variants.
+
+For an interactive walkthrough that exercises Boolean gates, the 8-bit ALU, mod-5 divisibility, and a CPU loop end-to-end:
+
+```bash
+python play.py                                            # 1 KB demo, runs in seconds
+python play.py --model neural_computer.safetensors        # 64 KB, slower
+```
+
+For end-to-end CPU validation (Fibonacci, sum 1..N, bubble sort, self-modifying JMP, all eight conditional jumps, CALL stack semantics, MUL cross-checked against repeated ADD, DIV cross-checked against repeated SUB, a bitwise AND/OR/XOR/SHL/SHR chain, and the architectural flag policy):
+
+```bash
+python test_cpu.py                                        # default: 1 KB, ~2 s
+python test_cpu.py --model neural_computer.safetensors    # 64 KB canonical, ~2 min
+python test_cpu.py --only fib,sum_n                       # subset of suite
+```
+
+Each program is assembled by a small Python assembler (`cpu_programs.py`) and run through the threshold-gated CPU; the driver verifies expected memory contents at HALT.
+
+---
+
+## Execution model
+
+A self-contained machine. State goes in, state comes out:
+
+- **Pure tensor computation**: state in, state out
+- **Frozen circuits**: integer weights, Heaviside activation
+- **ACT execution**: internal loop until `HALT`
+- **No external orchestration**: one forward pass equals one complete program execution
+
+Every datapath operation runs through threshold gates: ALU arithmetic
+(ADD/SUB via ripple-carry, MUL via partial-product AND gates and shift-add,
+DIV via per-stage bit-cascade comparators), bitwise logic, shifts,
+comparisons, conditional-jump PC muxing, stack-pointer stepping, and all
+memory reads and writes. Instruction decode (bit-field selection), PC
+sequencing, and the bit complements feeding negated-condition selects are
+fixed wiring in the runtimes — structural routing, not gates.
+
+```
+            ┌─────────────────────────────┐
+            │      Initial State          │
+            │  [PC|Regs|Flags|Memory...]  │
+            └─────────────┬───────────────┘
+                          ▼
+            ┌─────────────────────────────┐
+            │   Threshold Circuit Layer   │
+            │  ┌───────────────────────┐  │
+            │  │   Fetch: PC → Instr   │  │
+            │  ├───────────────────────┤  │
+            │  │   Decode: Opcode/Ops  │  │
+            │  ├───────────────────────┤  │
+            │  │   Execute: ALU/Mem    │  │
+            │  ├───────────────────────┤  │
+            │  │   Writeback: Results  │  │
+            │  ├───────────────────────┤  │
+            │  │   PC Update           │  │
+            │  └───────────┬───────────┘  │
+            │              │              │
+            │         ┌────▼────┐         │
+            │         │ HALTED? │         │
+            │         └────┬────┘         │
+            │       no ────┴──── yes      │
+            │        │           │        │
+            │        ▼           ▼        │
+            │     [loop]      [exit]      │
+            └─────────────┬───────────────┘
+                          ▼
+            ┌─────────────────────────────┐
+            │       Final State           │
+            └─────────────────────────────┘
+```
+
+### Instruction set
+
+| Opcode | Mnemonic | Operation |
+|--------|----------|-----------|
+| 0x0 | ADD | R[d] = R[a] + R[b] |
+| 0x1 | SUB | R[d] = R[a] - R[b] |
+| 0x2 | AND | R[d] = R[a] & R[b] |
+| 0x3 | OR | R[d] = R[a] \| R[b] |
+| 0x4 | XOR | R[d] = R[a] ^ R[b] |
+| 0x5 | SHL | R[d] = R[a] << 1 |
+| 0x6 | SHR | R[d] = R[a] >> 1 |
+| 0x7 | MUL | R[d] = R[a] * R[b] |
+| 0x8 | DIV | R[d] = R[a] / R[b] |
+| 0x9 | CMP | flags = R[a] - R[b] |
+| 0xA | LOAD | R[d] = M[addr] |
+| 0xB | STORE | M[addr] = R[s] |
+| 0xC | JMP | PC = addr |
+| 0xD | Jcc | PC = addr if cond (imm8[2:0]: 0=Z, 1=NZ, 2=C, 3=NC, 4=N, 5=P, 6=V, 7=NV) |
+| 0xE | CALL | push PC; PC = addr |
+| 0xF | HALT | stop execution |
+
+**Flag policy.** Only ADD, SUB, MUL, and CMP write the FLAGS register; AND,
+OR, XOR, SHL, SHR, DIV, LOAD, STORE, and all control transfers leave it
+unchanged. Branch on the arithmetic op that set the condition — intervening
+bitwise or shift instructions do not disturb it. The `flags_policy` program
+in the CPU suite pins this behavior.
+
+| Flag | ADD | SUB / CMP | MUL |
+|---|---|---|---|
+| Z | result == 0 | result == 0 | result == 0 |
+| N | result bit 7 | result bit 7 | result bit 7 |
+| C | carry-out | 1 when no borrow (`a >= b`) | 0 |
+| V | signed overflow | signed overflow | 0 |
+
+### State tensor layout
+
+The **state tensor** uses MSB-first bit ordering: index 0 of each multi-bit field is the most-significant bit. So `R0[0]` is bit 7 of the architectural register, `R0[7]` is bit 0.
+
+```
+[ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ]
+```
+
+`N` is the address width (configurable, 0–16). Flags are ordered `Z, N, C, V`. Control bits are ordered `HALT, MEM_WE, MEM_RE, RESERVED`.
+
+#### Bit ordering, one rule per scope
+
+The state tensor's MSB-first convention does **not** propagate to subcircuit ports. Each subcircuit names its operand bits in its own scope:
+
+| Scope | Convention | Example |
+|---|---|---|
+| State tensor | MSB-first (index 0 = MSB) | `R0[0]` is bit 7 of register R0 |
+| Integer subcircuit ports (`$a[i]`, `$b[i]`) | LSB-indexed (index 0 = LSB) | `$a[0]` is bit 0 of operand `a` |
+| Ripple-carry full adders (`fa0..fa7`) | LSB-first (fa0 = LSB) | `fa0` consumes `$a[0]` and `$b[0]` |
+| Composed float circuits (`float16.add` etc.) | MSB-first operand words | `$a[0]` is the sign bit, `$a[1..E]` the exponent |
+| Float result gates (`exp_out.bit{k}`, `frac_out.bit{k}`) | LSB-first field bits | `frac_out.bit0` is the fraction LSB |
+| Instruction word | MSB-first (bit 15 = opcode high) | bit 15 is `opcode[3]` |
+
+Worked example for `arithmetic.ripplecarry8bit`:
+
+- Inputs: `$a[0]..$a[7]` and `$b[0]..$b[7]` where `$a[0]` is the LSB of `a`. To add `a = 0x05 = 0b00000101` and `b = 0x03`, drive `a[0]=1, a[1]=0, a[2]=1` (rest 0) and `b[0]=1, b[1]=1` (rest 0).
+- Outputs: `fa0.ha2.sum.layer2`..`fa7.ha2.sum.layer2` are sum bits 0..7 (LSB to MSB), and `fa7.carry_or` is the final carry-out. The 8-bit result is `{fa7..fa0}` reading high-to-low.
+
+This is also how `safetensors2verilog`'s threshold-logic frontend exposes the ports of any extracted subcircuit; `python -m safetensors2verilog ... --inspect` prints the port contract for any extracted circuit.
+
+### Instruction encoding (16-bit, MSB-first)
+
+```
+15..12  11..10  9..8  7..0
+opcode  rd      rs    imm8
+```
+
+Interpretation:
+- **R-type**: `rd = rd op rs` (imm8 ignored)
+- **I-type**: `rd = op rd, imm8` (rs ignored)
+- **Address-extended**: `LOAD`, `STORE`, `JMP`, `Jcc`, `CALL` consume the next word as a 16-bit address (big-endian); `imm8` is reserved and the PC skips 4 bytes when the jump is not taken.
+
+### Circuit categories
+
+| Category | Circuits | Examples |
+|----------|----------|----------|
+| Boolean | 9 | AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES |
+| Arithmetic | 18+ | half/full adder, ripple-carry (8/16/32-bit), comparators (8/16/32-bit), 3-operand adder, A+B×C and (A+B)×C expressions |
+| ALU | 8/16/32-bit | shifts, multiply, divide, INC/DEC, NEG, ROL/ROR, bitwise |
+| Combinational | 10+ | MUX (2:1, 4:1, 8:1), DEMUX, 3-to-8 decoder, 8-to-3 encoder, barrel shifter, priority encoder |
+| Control flow | 16 | JMP, conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV), CALL, RET, PUSH, POP |
+| Memory | 3 | N-bit address decoder, read mux, write cells (packed) |
+| Modular | 11 | divisibility by 2–12 (multi-layer for non-powers-of-2) |
+| Threshold | 13 | k-of-n gates, majority, minority, exactly-k |
+| Pattern | 10 | popcount, leading/trailing ones, symmetry |
+| Error detection | 11 | parity (XOR tree), checksum, CRC, Hamming |
+| Float (IEEE 754) | half + single | composed ADD, MUL, DIV, EQ/LT/LE/GT/GE pipelines plus unpack/pack/classify/normalize stages |
+
+The float ADD/MUL/DIV/CMP circuits are **self-contained composed pipelines**: their complete internal wiring ships in the files as `.inputs` metadata, their external inputs are the raw operand words (`$a[0]` sign, `$a[1..E]` exponent, then mantissa; MSB-first), and the eval suite reconstructs and executes each netlist end to end from that metadata alone (`NetlistEvaluator`). Comparisons are fully IEEE (NaN unordered, `+0 == -0`, subnormal ordering, mixed signs). The arithmetic is round-to-nearest-even, **bit-exact to IEEE hardware** on the normal range, with exact specials (NaN, infinities including `inf - inf`/`inf * 0`/`0/0`/`inf/inf` → NaN and `x/0` → inf, signed zeros) and flush-to-zero for subnormal operands and results. Subnormal operands and results currently flush to zero; full subnormal support (gradual underflow) is unfinished and tracked in `todo.md`. Because the tests run from each file's own routing metadata, they prove the artifact is self-contained: `safetensors2verilog`-style extraction of `float16.add` and friends has everything it needs to emit a single composed module.
+
+### Tensor naming
+
+```
+{category}.{circuit}[.{layer}][.{component}].{weight|bias}
+
+Examples:
+  boolean.and.weight
+  boolean.xor.layer1.neuron1.weight
+  arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight
+  modular.mod5.eq.k15.bit3.match.weight
+  error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias
+```
+
+Memory circuits are stored as packed tensors so the safetensors header stays manageable (`memory.addr_decode.weight`, `memory.read.and.weight`, `memory.write.and_old.weight`, etc.).
+
+---
+
+## Bit widths and memory profiles
+
+The build tool emits one of 51 functionally distinct configurations: three data-path widths × seventeen address widths (0–16, where 0 means no memory).
+
+**Bit widths** (`--bits`):
+
+| Width | Range | Use case |
+|-------|-------|----------|
+| 8 | 0–255 | full CPU, legacy compatibility |
+| 16 | 0–65,535 | extended arithmetic |
+| 32 | 0–4,294,967,295 | practical arithmetic ranges |
+
+**Memory profiles** (`-m`):
+
+| Profile | Size | Addr bits | Filename suffix |
+|---------|------|-----------|-----------------|
+| `none` | 0 B | 0 | (uses `alu` instead of `computer`) |
+| `registers` | 16 B | 4 | `_registers` |
+| `scratchpad` | 256 B | 8 | `_scratchpad` |
+| `small` | 1 KB | 10 | `_small` |
+| `reduced` | 4 KB | 12 | `_reduced` |
+| `full` | 64 KB | 16 | (none) |
+
+Auto-generated filename: `neural_{alu|computer}{BITS}[_{MEMORY}].safetensors`. Custom address widths via `-a N` produce `_addrN`.
+
+```bash
+python build.py --bits 32 --apply all              # neural_computer32.safetensors
+python build.py --bits 8 -m none --apply all       # neural_alu8.safetensors
+python build.py --bits 16 -m small --apply all     # neural_computer16_small.safetensors
+python build.py --bits 32 -a 6 --apply all         # neural_computer32_addr6.safetensors
+```
+
+To regenerate every named variant in one pass:
+
+```bash
+python build_all.py
+```
+
+This populates `variants/` with all 18 builds, quantizes each one to the smallest signed integer dtype that exactly represents its weights (~4× reduction in tensor data, with file size dominated by the safetensors header on the smaller profiles), verifies the strictly ternary weight invariant (`--ternary --strict`, so a build with any non-ternary weight fails loudly), stamps the `weight_quantization` metadata field, and runs `eval.py` on each as a sanity check.
+
+The quantizer is also available standalone:
+
+```bash
+python quantize.py path/to/file.safetensors           # in-place
+python quantize.py variants/                          # whole directory
+python quantize.py model.safetensors -o quantized.safetensors
+python quantize.py file.safetensors --ternary         # push toward {-1, 0, 1} weights
+python quantize.py file.safetensors --ternary --strict  # error if any weight is non-ternary
+```
+
+Every weight and bias tensor in the canonical model fits in `int8`. The eval pipeline promotes weights to `float32` on load, so integer storage is exact and transparent.
+
+**Ternary mode.** `build.py` emits only ternary weights: identity buffers are `weight=1, bias=-1` (`H(x - 1)`), and the comparators, modular detectors, and division stages that previously required positional weights up to ±2³¹ are bit-cascaded multi-layer equivalents. With `--ternary`, the quantizer verifies this and repairs legacy files: it rewrites historical single-input `weight=±2` buffers as `weight=±1` with the bias adjusted to preserve the heaviside output for binary inputs (`H(2x - 1) ≡ H(x - 1)`), and rebuilds pre-bit-cascade modular detectors (moduli already in bit-cascade form are left untouched, routing metadata included). `--strict` fails if any weight tensor remains non-ternary. Every shipped file carries the metadata field `weight_quantization: ternary`; a repaired file with remaining violations would be stamped `ternary_partial`.
+
+---
+
+## Verification
+
+| Category | Coverage | Notes |
+|----------|----------|-------|
+| Boolean gates | exhaustive | all 2^n input combinations |
+| Arithmetic (8-bit) | strategic sampling | edge values + diagonal pairs; ~50 cases per circuit |
+| Arithmetic (16/32-bit) | strategic sampling | width-scaled extremes, alternating patterns, byte-boundary carries |
+| ALU primitives (8/16/32-bit) | strategic sampling | edge inputs per operation; DIV comparators driven along real restoring-division traces |
+| Control flow | exhaustive | all 2^3 input combinations per Jcc, per address bit |
+| Threshold k-of-n | exhaustive | all 256 8-bit popcounts |
+| Modular (all moduli, 8-bit input) | exhaustive | every value in [0, 255] |
+| Parity | exhaustive | every value in [0, 255] |
+| Pattern recognition | exhaustive | every value in [0, 255] |
+| Combinational logic | exhaustive | full input space per gate |
+| Float unpack/pack | exhaustive, functional | every bit gate driven with 0 and 1 (identity) |
+| Float classify | functional | IEEE 754 categories (zero, subnormal, normal, inf, NaN) at edge encodings, both widths |
+| Float CMP (composed) | functional, exact IEEE | full netlist rebuilt from the shipped `.inputs` metadata and evaluated end to end; NaN unordered, signed zeros, subnormal ordering, mixed signs — all five predicates, both widths |
+| Float ADD/MUL/DIV (composed) | functional, bit-exact to IEEE hardware | same metadata-driven evaluation: exact specials, flush-to-zero, round-to-nearest-even; cross-checked against native float arithmetic |
+| Memory / manifest | structure checks | packed-tensor shapes against the manifest |
+| CPU integration | program-level | ten assembled programs (Fibonacci, sum, sort, self-modifying JMP, all eight Jcc, CALL stack push, MUL vs repeated ADD, DIV vs repeated SUB, a bitwise AND/OR/XOR/SHL/SHR pipeline, and the flag-policy pin) |
+
+The 8-bit arithmetic and ALU tests use strategic sampling rather than the full 65,536-case sweep because exhaustive coverage at 8-bit is feasible but not necessary given that the circuits are constructed gate-by-gate. The 16-bit and 32-bit arithmetic tests sample edge cases only; full exhaustive coverage at those widths is infeasible without specialized hardware.
+
+`eval_all.py` runs the unified suite. Exit code is the number of failing variants (0 means all pass). **Testing is evaluation, not rebuilding**: `python eval_all.py variants/` scores all 18 fitness variants against the shipped weights in about two minutes (~6 s each, the composed float netlists evaluated in `NetlistEvaluator`'s leveled mode) and cleanly skips the two standalone machines. Rebuilding the models (`build_all.py`, ~50 min for all 18) is a separate step, needed only when the circuit constructions in `build.py` change — routine verification never rebuilds. The batched evaluator is population-safe: every chained intermediate (carry, borrow, mux select) is computed per population slot, so `prune_weights.py`'s parallel fitness screens are exact rather than slot-0 approximations.
+
+---
+
+## neural_subleq8 — the one-instruction machine
+
+The minimal member of the family: a Turing-complete computer with **no instruction decoder** (there is only one instruction), **no registers**, and **no flags** — 194 logic gates totalling 548 ternary parameters. Its one architectural decision, branch or fall through, is a single threshold neuron (`subleq.leq`) over the sign and zero of the subtraction result.
+
+- 8-bit data, 8-bit addresses, 256 bytes of packed threshold memory, a program counter and nothing else.
+- An instruction is three bytes `A B C`. Step: `M[B] = (M[B] - M[A]) mod 256`; if the result is `<= 0` in two's complement (zero or bit 7 set), `PC = C`, else `PC += 3`. `PC = 0xFF` halts.
+- Circuits (all wiring shipped as `.inputs` metadata): an 8-bit two's-complement subtractor, the branch neuron, a `PC + 3` incrementer, a branch mux, and the packed 256-row memory.
+
+```bash
+python build.py --apply subleq          # -> variants/neural_subleq8.safetensors
+python machines.py subleq               # exhaustive datapath + lockstep programs
+```
+
+`machines.py subleq` evaluates the datapath **from the shipped wiring metadata** over all 65,536 operand pairs (result and branch decision both exhaustive), checks the PC mux, and runs a program suite (clear, negate-copy, add-by-double-negation, countdown loop) in full-state lockstep against a reference emulator. Third-party SUBLEQ toolchains target this machine directly once the `0xFF` halt convention is mapped.
+
+---
+
+## neural_rv32 — a RISC-V processor as a threshold network
+
+The most capable member: a RISC-V CPU whose entire datapath is ternary threshold gates (`variants/neural_rv32.safetensors`, ~8.6 M parameters, 64 KB memory, strictly ternary).
+
+- **RV32I base**: LUI, AUIPC, JAL, JALR, all six branches, LB/LH/LW/LBU/LHU, SB/SH/SW, and the full OP-IMM/OP groups. 32 × 32-bit registers (x0 zero), little-endian memory through the packed threshold circuits. ECALL halts.
+- **M extension**: MUL, MULH, MULHSU, MULHU (full 64-bit product through a shift-add array with gate-level sign correction), DIV, DIVU, REM, REMU (32 restoring stages, spec-exact divide-by-zero and overflow).
+- **F subset**: FLW, FSW, FMV.W.X, FMV.X.W, FADD.S, FSUB.S, FMUL.S, FDIV.S, FEQ.S, FLT.S, FLE.S, FSGNJ[N/X].S, FCVT.W.S, FCVT.S.W — the arithmetic executed by the composed float32 pipelines (round-to-nearest-even, bit-exact to hardware; specials and flush-to-zero as above), the int/float conversions gate-routed through the priority encoder and barrel shifter and cross-checked bit-exact against native conversion.
+- **NEUR (custom-0)**: `neur rd, rs1, rs2` evaluates one threshold neuron — `rd = H(popcount(rs1[7:0] & rs2[7:0]) - popcount(rs1[7:0] & rs2[15:8]) + sext(rs2[20:16]))`. Networks of NEUR instructions are neural networks running as software on the neural network; the test suite computes XOR with a two-layer NEUR net.
+- **Dual issue**: two adjacent OP/OP-IMM/LUI/AUIPC instructions retire in one cycle when the gate-level hazard comparators (`rv32.hazard.*`) clear RAW and WAW dependences.
+- **MMIO**: stores to `0xFF00` append a character to the console.
+
+Signed comparisons ride the unsigned bit-cascade with sign bits complemented through NOT gates; SLL uses the barrel shifter, SRL is bit-reversal wiring over it, SRA a gate mux over the complement form. Instruction decode, immediate extraction, register-file indexing, and PC sequencing are fixed wiring, per the family convention.
+
+```bash
+python build.py --apply rv32                            # build the file
+python quantize.py variants/neural_rv32.safetensors --ternary --strict
+python machines.py rv32                                 # eight-program lockstep suite
+python machines.py rv32-c                               # stock-compiler C, end to end
+```
+
+**Running compiled C.** `machines.py rv32-c` compiles a freestanding C program (gcd, Fibonacci, insertion sort; `rv32im`, so real `mul`/`rem`) with an unmodified clang rv32im toolchain, loads the relocatable object with an in-repo loader (no external linker — it resolves the R_RISCV relocations of one translation unit and lays the sections out flat), executes it on the threshold CPU, and checks the return value against the value computed natively. The program retires in ~300 instructions and matches exactly. Stock `rv32im` toolchains (gcc, clang, rustc) emit this ISA. Unfinished float work — subnormals, FMA, and the FCVT rounding-mode field — is tracked in `todo.md`.
+
+The composed circuits evaluate in a leveled mode (`NetlistEvaluator`, one padded tensor op per topological level instead of one Python step per gate), ~18× faster on the FPU-scale netlists.
+
+---
+
+## Threshold logic
+
+A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing the result to a threshold; the output is 1 when the sum meets or exceeds the threshold and 0 otherwise. Equivalently, it is a neuron with Heaviside step activation, integer weights, and an integer bias.
+
+Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function, which includes AND, OR, NAND, NOR, IMPLIES, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (XOR, parity, mod-k for k not a power of two) require multiple layers.
+
+Example gates:
+
+```
+AND: w=[1, 1], b=-2
+  H(0+0-2) = 0     H(1+1-2) = 1
+
+OR:  w=[1, 1], b=-1
+  H(0+0-1) = 0     H(1+0-1) = 1
+
+XOR: two layers (not linearly separable)
+  layer 1: OR + NAND
+  layer 2: AND of the two
+```
+
+A full adder is two half-adders plus a carry OR, around four threshold layers. An 8-bit ripple-carry adder is eight chained full adders, around 32 layers.
+
+### History
+
+Warren McCulloch and Walter Pitts introduced the threshold neuron in 1943, proving that networks of such neurons can compute any Boolean function. Their work preceded both the perceptron and modern neural networks and established the theoretical foundation for neural computation.
+
+The 1960s saw substantial work on threshold logic synthesis. Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function can be implemented as a single threshold gate, and if so, how to compute the appropriate weights. The focus was on individual gates rather than complete systems.
+
+Frank Rosenblatt's Mark I Perceptron (1957–1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training; the final configurations were not published. Bernard Widrow's ADALINE and MADALINE (1960–1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm.
+
+Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing-complete. The construction relied on continuous sigmoid activations with infinite precision, not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: universality with continuous activations chosen to support gradient-based training.
+
+### Neuromorphic hardware
+
+Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon:
+
+- **Intel Loihi** (2017): 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. Supports integer weights and configurable neuron dynamics.
+- **IBM TrueNorth** (2014): one million neurons and 256 million synapses across a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds.
+- **BrainChip Akida** (2021): edge-oriented event-based processing with integer weights.
+- **SpiNNaker** (University of Manchester): ARM cores simulating spiking networks at scale.
+
+Published work on these platforms has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1 and described that work as "a first step toward the construction of a spiking computer"; that implementation lacked instruction fetch, a program counter, memory, and control logic.
+
+The weights in this repository implement a complete CPU: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every logic component is a threshold neuron with integer weights; instruction decode and PC sequencing are fixed wiring between them.
+
+---
+
+## Hardware compatibility
+
+All weights are in {-1, 0, 1}, all activations are Heaviside step, and every gate is a single weighted sum followed by a sign test. This eliminates multipliers entirely: each gate evaluation is a popcount of `+1`-weighted inputs minus a popcount of `-1`-weighted inputs plus an integer bias. The circuits are intended to deploy directly on:
+
+- **FPGA**: every gate maps to a small LUT cluster (or a popcount tree of LUT4/LUT6 + carry chain). Ternary weight storage compresses to 2 bits per weight; routing collapses to bit selection.
+- **Intel Loihi**: integer weights and Heaviside threshold neurons are the native primitive. Ternary fits well within Loihi's 8-bit weight range.
+- **IBM TrueNorth**: configurable threshold per neurosynaptic core; ternary weights and small biases are within the supported range.
+- **BrainChip Akida**: edge-oriented integer-weight inference; ternary weights fit cleanly.
+
+---
+
+## LLM integration
+
+Threshold circuits can be embedded into transformer MLP layers to give a language model exact arithmetic. Standard LLMs fail at arithmetic because they interpolate over the training distribution rather than compute, so a 360M-parameter model trained on web text has seen `127 + 128 = 255` few times if at all and guesses based on pattern matching.
+
+The integration freezes the circuits and trains only the interface layers that:
+
+1. Extract operands from token embeddings.
+2. Route computation through the appropriate circuit.
+3. Inject the result back into the residual stream.
+
+The model learns *call dispatch*; the arithmetic is already solved.
+
+### Architecture
+
+```
+x ──┬── MLP path ─────────────────┬── + ── output
+    │                             │
+    └── BitExtractor ── Circuit ──┴── BitInjector
+                          │
+                       Router (learned weighting)
+```
+
+Augmented MLP forward pass:
+
+```python
+def forward(x):  # x: [batch, seq, d_model=960]
+    mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x))
+
+    a_bits, b_bits = self.bit_extractor(x)              # [batch, seq, 8] each
+    result_bits, carry = self.circuits.add_8bit(a_bits, b_bits)
+    flags = self.compute_flags(result_bits, carry)
+    circuit_delta = self.bit_injector(result_bits, flags)
+
+    route_weights = self.router(x)                       # [batch, seq, 2] softmax
+    return mlp_out + route_weights[..., 1:2] * circuit_delta
+```
+
+### Target model
+
+The reference integration uses HuggingFace's [SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct). See [`llm_integration/SMOLLM2_ARCHITECTURE.md`](llm_integration/smollm2/SMOLLM2_ARCHITECTURE.md) for the full technical analysis.
+
+| Property | Value |
+|----------|-------|
+| Parameters | 361.82 M |
+| Hidden dimension | 960 (matches the extractor input) |
+| Layers | 32 transformer blocks |
+| Attention | 15 query heads, 5 KV heads (GQA) |
+| MLP | SwiGLU (960 → 2560 → 960) |
+| Position encoding | RoPE (theta = 100k, max 8192) |
+
+Digits tokenize individually (`"47 + 86"` → `['4', '7', ' +', ' ', '8', '6']`, with digit token IDs `32 + digit_value`), which makes position-based operand extraction practical.
+
+### Gradient flow
+
+Heaviside has zero gradient almost everywhere. The implementation uses a straight-through estimator:
+
+```python
+class HeavisideSTE(torch.autograd.Function):
+    @staticmethod
+    def forward(ctx, x):
+        return (x >= 0).float()
+
+    @staticmethod
+    def backward(ctx, grad_output):
+        return grad_output
+```
+
+At inference, Heaviside is the true step function; if the extractor identifies operands correctly, the circuit produces the correct result by construction.
+
+### Baseline
+
+SmolLM2-360M-Instruct on randomized 8-bit arithmetic (2,000 cases, operands uniform on [0, 255], generous answer extraction):
+
+| Operation | Accuracy |
+|-----------|----------|
+| Addition | 35.92% |
+| Subtraction | 17.72% |
+| Multiplication | 1.25% |
+| Greater than | 14.37% |
+| Less than | 4.31% |
+| Equality | 0.28% |
+| **Overall** | **11.90%** (238/2000) |
+
+Multiplication accuracy at 1.25% is essentially random over the output space. Comparison operations often echo the expression rather than evaluate it. Even addition fails roughly two-thirds of the time on full 8-bit operands. Performance degrades further as operand magnitude increases: edge cases like `127 + 128` are almost never correct.
+
+The frozen threshold circuits reach 100% on the same task when given correctly formatted bit inputs (10,000 random cases, every operation). The integration challenge is therefore the extractor, not the arithmetic.
+
+### Trainable parameters (SmolLM2, hidden_dim = 960)
+
+| Component | Parameters | Description |
+|-----------|------------|-------------|
+| AttentionPooling | ~3.7 M | 4-head attention over the sequence |
+| MultiHeadBitExtractor (× 2) | ~245 K each | 8 per-bit MLPs for A and B |
+| OpRouter | ~246 K | 960 → 256 → 6 MLP |
+| **Extractor total** | **~4.4 M** | full extraction module |
+
+Alternative architectures: `PositionExtractor` (~1.5 M, position-specific, no attention), `DigitExtractor` (~1.2 M, predicts digits 0–9 instead of bits), `HybridExtractor` (digit lookup with MLP fallback for word numerals). With `--unfreeze_layers 4` an additional ~39.3 M trainable parameters open up in the top four transformer layers.
+
+### Training
+
+```bash
+python train.py --mode router --epochs 100                          # sanity check
+python train.py --mode llm --epochs 100 --batch_size 256            # frozen LLM
+python train.py --mode llm --unfreeze_layers 4 --batch_size 4096    # fine-tune top layers
+```
+
+Loss components: BCE on output bits, BCE on extracted A and B bits (2× weight), and CE on operation classification. Curriculum runs 0–9 → 0–99 → 0–255. Optimizer is AdamW, lr 3e-4, gradient clipping 1.0.
+
+---
+
+## Repository layout
+
+```
+neural_computer.safetensors         canonical model (32-bit, 64 KB, ~8.61M params)
+variants/                           18 prebuilt configurations + neural_subleq8 + neural_rv32
+build.py                            generator (one safetensors per invocation; also `subleq`, `rv32`)
+build_all.py                        builds, quantizes (--ternary --strict), and verifies every profile
+quantize.py                         min integer dtypes + ternary verification/repair
+eval.py                             gate-level fitness suite, NetlistEvaluator, 64 KB reference CPU
+eval_all.py                         variant-agnostic harness + manifest-sized threshold CPU
+cpu_programs.py                     assembler + ten-program suite for CPU-level validation
+test_cpu.py                         runs the program suite against a chosen variant
+machines.py                         neural_subleq8 + neural_rv32 runtimes, references, assemblers,
+                                    RV32 object loader, and both test suites (subleq / rv32 / rv32-c)
+play.py                             interactive demo
+prune_weights.py                    GPU-batched weight reduction with conflict resolution
+llm_integration/                    SmolLM2 extractor + circuit wrapper + training code
+  ├── circuits.py                   FrozenThresholdCircuits (loads safetensors, exposes
+  │                                 add_8bit / sub_8bit / mul_8bit / compare_*)
+  ├── model.py                      Extractor variants, ArithmeticModel
+  ├── train.py                      router / interface / llm training modes
+  ├── fitness.py                    randomized fitness function
+  ├── baseline.py                   vanilla SmolLM2 baseline measurement
+  ├── trained/                      checkpointed extractor weights
+  └── smollm2/
+      ├── SMOLLM2_ARCHITECTURE.md   architecture analysis
+      ├── analyze_smollm2.py        analysis script
+      └── smollm2_analysis.json     analysis output
+```
+
+---
+
+## Citation
+
+```bibtex
+@misc{8bit-threshold-computer,
+  title={8bit-threshold-computer: A Turing-Complete Threshold Logic CPU},
+  author={Norton, Charles},
+  year={2026},
+  howpublished={Hugging Face},
+  url={https://huggingface.co/phanerozoic/8bit-threshold-computer}
+}
+```
+
+---
+
+## License
+
+MIT
+
+---
+
+## References
+
+1. McCulloch & Pitts (1943). *A Logical Calculus of Ideas Immanent in Nervous Activity.*
+2. Muroga (1971). *Threshold Logic and Its Applications.*
+3. Siegelmann & Sontag (1995). *On the Computational Power of Neural Nets.*
+4. Bengio et al. (2013). *Estimating or Propagating Gradients Through Stochastic Neurons.*
+5. Ma et al. (2024). *The Era of 1-bit LLMs* (BitNet b1.58).
+6. HuggingFace (2024). *SmolLM2: Small Language Models* — [model card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct).
+7. Vaswani et al. (2017). *Attention Is All You Need.*
+8. Su et al. (2021). *RoFormer: Enhanced Transformer with Rotary Position Embedding.*

build_all.py

@@ -0,0 +1,214 @@
+"""
+Build and verify every named (bits, memory_profile) variant.
+
+Outputs:
+  variants/neural_alu{8,16,32}.safetensors                          - no memory
+  variants/neural_computer{8,16,32}_registers.safetensors           - 16 B
+  variants/neural_computer{8,16,32}_scratchpad.safetensors          - 256 B
+  variants/neural_computer{8,16,32}_small.safetensors               - 1 KB
+  variants/neural_computer{8,16,32}_reduced.safetensors             - 4 KB
+  variants/neural_computer{8,16,32}.safetensors                     - 64 KB
+
+Each variant is built from the canonical seed, quantized to minimum
+integer dtypes with the strict ternary check (--ternary --strict), and
+then verified with eval.py via the BatchedFitnessEvaluator; the summary
+records (tensor count, params, file size, fitness, total_tests, seconds).
+"""
+
+from __future__ import annotations
+import os
+import shutil
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+import torch
+from safetensors import safe_open
+
+ROOT = Path(__file__).resolve().parent
+SEED = ROOT / "neural_computer.safetensors"
+OUT_DIR = ROOT / "variants"
+OUT_DIR.mkdir(exist_ok=True)
+
+PROFILES = ["none", "registers", "scratchpad", "small", "reduced", "full"]
+BITS = [8, 16, 32]
+
+
+def variant_filename(bits: int, profile: str) -> str:
+    if profile == "none":
+        return f"neural_alu{bits}.safetensors"
+    if profile == "full":
+        return f"neural_computer{bits}.safetensors"
+    return f"neural_computer{bits}_{profile}.safetensors"
+
+
+def run(cmd: list[str], timeout: int = 600) -> tuple[int, str]:
+    p = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
+    return p.returncode, (p.stdout or "") + (p.stderr or "")
+
+
+def build_variant(bits: int, profile: str) -> Path:
+    out = OUT_DIR / variant_filename(bits, profile)
+    shutil.copy2(SEED, out)
+    cmd = [
+        sys.executable, str(ROOT / "build.py"),
+        "--bits", str(bits),
+        "-m", profile,
+        "--apply",
+        "--model", str(out),
+        "all",
+    ]
+    rc, log = run(cmd, timeout=900)
+    if rc != 0:
+        raise RuntimeError(f"build failed for bits={bits} profile={profile}:\n{log[-1500:]}")
+    return out
+
+
+def quantize_variant(path: Path) -> tuple[int, int]:
+    """Run quantize.py --ternary --strict on a built variant. This is the
+    last pipeline step: it casts every tensor to its minimum signed integer
+    dtype, verifies the strictly ternary weight invariant, and stamps the
+    weight_quantization metadata field. Returns (bytes_before, bytes_after).
+    """
+    rc, log = run([sys.executable, str(ROOT / "quantize.py"), str(path),
+                   "--ternary", "--strict"], timeout=300)
+    if rc != 0:
+        raise RuntimeError(f"quantize failed for {path.name}:\n{log[-800:]}")
+    # parse the "file X.X MB -> Y.Y MB" line
+    for line in log.splitlines():
+        if "file" in line and "->" in line and path.name in line:
+            try:
+                parts = line.split("file")[1].split("->")
+                before = float(parts[0].strip().split()[0]) * 1e6
+                after = float(parts[1].strip().split()[0]) * 1e6
+                return int(before), int(after)
+            except Exception:
+                pass
+    return 0, 0
+
+
+def measure_variant(path: Path) -> dict:
+    """Read tensor count, params, manifest values from the variant."""
+    with safe_open(str(path), framework="pt") as f:
+        keys = list(f.keys())
+        params = sum(f.get_tensor(k).numel() for k in keys)
+        manifest = {
+            k.split(".", 1)[1]: f.get_tensor(k).item()
+            for k in keys if k.startswith("manifest.") and f.get_tensor(k).numel() == 1
+        }
+    return {
+        "tensors": len(keys),
+        "params": params,
+        "size_mb": path.stat().st_size / (1024 * 1024),
+        "manifest": manifest,
+    }
+
+
+def eval_variant(path: Path, device: str = "cpu", timeout: int = 600) -> dict:
+    """Run eval.py against a variant and parse fitness."""
+    cmd = [
+        sys.executable, str(ROOT / "eval.py"),
+        "--model", str(path),
+        "--device", device,
+        "--quiet",
+    ]
+    t0 = time.time()
+    rc, log = run(cmd, timeout=timeout)
+    dt = time.time() - t0
+
+    fitness = None
+    total_tests = None
+    status = "ERROR"
+    for line in log.splitlines():
+        line = line.strip()
+        if line.startswith("Fitness:"):
+            try:
+                fitness = float(line.split()[1])
+            except Exception:
+                pass
+        elif line.startswith("Total tests:"):
+            try:
+                total_tests = int(line.split()[2])
+            except Exception:
+                pass
+        elif line.startswith("STATUS:"):
+            status = line.split()[1]
+    return {
+        "rc": rc,
+        "fitness": fitness,
+        "total_tests": total_tests,
+        "status": status,
+        "elapsed_s": dt,
+        "log_tail": "\n".join(log.splitlines()[-15:]),
+    }
+
+
+def main() -> None:
+    rows = []
+    print(f"Building 18 variants into {OUT_DIR}\n")
+    for bits in BITS:
+        for profile in PROFILES:
+            label = f"bits={bits} profile={profile}"
+            print(f"=== {label} ===", flush=True)
+            t0 = time.time()
+            try:
+                path = build_variant(bits, profile)
+                bt = time.time() - t0
+                pre_q_meta = measure_variant(path)
+                # Quantize in-place as the final step; weights are
+                # integer-valued so this is exact, --strict fails the build
+                # if any weight is non-ternary, and header metadata
+                # (signal_registry) is carried through.
+                qb, qa = quantize_variant(path)
+                meta = measure_variant(path)
+                ev = eval_variant(path, device="cpu", timeout=900)
+                rows.append({
+                    "bits": bits, "profile": profile,
+                    "filename": path.name,
+                    "build_s": bt,
+                    **meta,
+                    **{k: ev[k] for k in ("fitness", "total_tests", "status", "elapsed_s")},
+                    "log_tail": ev["log_tail"] if ev["status"] != "PASS" else "",
+                })
+                q_ratio = qb / qa if qa else 1.0
+                print(f"  built in {bt:.1f}s  size={pre_q_meta['size_mb']:.1f}MB -> "
+                      f"{meta['size_mb']:.1f}MB after quant ({q_ratio:.2f}x)"
+                      f"  params={meta['params']:,}  tensors={meta['tensors']:,}")
+                print(f"  eval: fitness={ev['fitness']}  tests={ev['total_tests']}"
+                      f"  status={ev['status']}  ({ev['elapsed_s']:.1f}s)")
+                if ev["status"] != "PASS":
+                    print("  --- failure tail ---")
+                    print("  " + "\n  ".join(ev["log_tail"].splitlines()))
+                    print("  --------------------")
+            except Exception as e:
+                print(f"  EXCEPTION: {e}")
+                rows.append({"bits": bits, "profile": profile, "error": str(e)})
+            print()
+
+    print("=" * 88)
+    print(" SUMMARY")
+    print("=" * 88)
+    header = f"{'bits':>4} {'profile':<11} {'size_MB':>8} {'tensors':>8} {'params':>11} {'fitness':>9} {'tests':>6} {'status':>7}"
+    print(header)
+    print("-" * len(header))
+    for r in rows:
+        if "error" in r:
+            print(f"{r['bits']:>4} {r['profile']:<11} ERROR: {r['error'][:60]}")
+            continue
+        fit = f"{r['fitness']:.4f}" if r['fitness'] is not None else "n/a"
+        tests = r['total_tests'] if r['total_tests'] is not None else "?"
+        print(f"{r['bits']:>4} {r['profile']:<11} {r['size_mb']:>8.1f} "
+              f"{r['tensors']:>8,} {r['params']:>11,} "
+              f"{fit:>9} {tests:>6} {r['status']:>7}")
+
+    fail = [r for r in rows if r.get("status") != "PASS" or "error" in r]
+    print()
+    if fail:
+        print(f"FAILURES: {len(fail)}/{len(rows)}")
+    else:
+        print(f"ALL {len(rows)} VARIANTS PASS")
+
+
+if __name__ == "__main__":
+    main()

cpu_programs.py

@@ -0,0 +1,703 @@
+"""
+CPU validation programs for the threshold computer.
+
+A small assembler and a suite of programs that exercise the ISA end-to-end:
+arithmetic, control flow, memory access, self-modifying code, all eight
+conditional jumps, the call mechanism, and a sort.
+
+Each program returns (mem, expected, max_cycles, description) where:
+  mem        : list[int] -- complete memory image
+  expected   : dict[int, int] -- {address: expected_value} verified at HALT
+  max_cycles : int -- cycle budget (an infinite loop will hit this)
+  description: str -- short human-readable summary
+
+Programs target the 1 KB profile (addr_bits=10) by default but use only the
+low 256 bytes so they also run on scratchpad and larger profiles.
+
+All programs assume the CPU starts with PC=0, SP defaulting to addr_mask
+(highest address; CALL pre-decrements before writing).
+"""
+
+from __future__ import annotations
+from typing import Dict, List, Tuple
+
+
+# ----------------------------------------------------------------------------
+# Mini assembler
+# ----------------------------------------------------------------------------
+
+_OPCODE_NAMES = {
+    "add": 0x0, "sub": 0x1, "and": 0x2, "or": 0x3, "xor": 0x4,
+    "shl": 0x5, "shr": 0x6, "mul": 0x7, "div": 0x8, "cmp": 0x9,
+    "load": 0xA, "store": 0xB, "jmp": 0xC, "jcc": 0xD,
+    "call": 0xE, "halt": 0xF,
+}
+
+_COND = {"jz": 0, "jnz": 1, "jc": 2, "jnc": 3,
+         "jn": 4, "jp": 5, "jv": 6, "jnv": 7}
+
+
+def _enc(opcode: int, rd: int = 0, rs: int = 0, imm: int = 0) -> int:
+    return ((opcode & 0xF) << 12) | ((rd & 0x3) << 10) | ((rs & 0x3) << 8) | (imm & 0xFF)
+
+
+class Asm:
+    """Tiny assembler for the threshold-computer ISA.
+
+    Usage:
+        a = Asm(size=256)
+        a.org(0)
+        a.label("start")
+        a.load(0, "data")
+        a.halt()
+        a.org(0x80); a.label("data"); a.db(42)
+        mem = a.assemble()
+    """
+
+    def __init__(self, size: int):
+        self.mem: List[int] = [0] * size
+        self.pc: int = 0
+        self.labels: Dict[str, int] = {}
+        self.fixups: List[Tuple[int, str]] = []
+
+    def org(self, addr: int) -> None:
+        self.pc = addr
+
+    def label(self, name: str) -> None:
+        if name in self.labels:
+            raise ValueError(f"duplicate label: {name}")
+        self.labels[name] = self.pc
+
+    def db(self, *values: int) -> None:
+        for v in values:
+            self.mem[self.pc] = v & 0xFF
+            self.pc += 1
+
+    def dw(self, value: int) -> None:
+        self.mem[self.pc] = (value >> 8) & 0xFF
+        self.mem[self.pc + 1] = value & 0xFF
+        self.pc += 2
+
+    def daddr(self, label: str) -> None:
+        self.fixups.append((self.pc, label))
+        self.dw(0)
+
+    # --- ALU ops (no immediate) ---
+    def _alu(self, op: int, rd: int, rs: int) -> None:
+        self.dw(_enc(op, rd, rs))
+
+    def add(self, rd: int, rs: int) -> None: self._alu(0x0, rd, rs)
+    def sub(self, rd: int, rs: int) -> None: self._alu(0x1, rd, rs)
+    def and_(self, rd: int, rs: int) -> None: self._alu(0x2, rd, rs)
+    def or_(self, rd: int, rs: int) -> None: self._alu(0x3, rd, rs)
+    def xor(self, rd: int, rs: int) -> None: self._alu(0x4, rd, rs)
+    def shl(self, rd: int) -> None: self._alu(0x5, rd, 0)
+    def shr(self, rd: int) -> None: self._alu(0x6, rd, 0)
+    def mul(self, rd: int, rs: int) -> None: self._alu(0x7, rd, rs)
+    def cmp(self, rd: int, rs: int) -> None: self._alu(0x9, rd, rs)
+
+    # --- Memory + control (address-extended) ---
+    def load(self, rd: int, label: str) -> None:
+        self.dw(_enc(0xA, rd, 0)); self.daddr(label)
+
+    def store(self, rs: int, label: str) -> None:
+        self.dw(_enc(0xB, 0, rs)); self.daddr(label)
+
+    def jmp(self, label: str) -> None:
+        self.dw(_enc(0xC)); self.daddr(label)
+
+    def jcc(self, cond: str, label: str) -> None:
+        self.dw(_enc(0xD, 0, 0, _COND[cond])); self.daddr(label)
+
+    def jz(self, label: str) -> None: self.jcc("jz", label)
+    def jnz(self, label: str) -> None: self.jcc("jnz", label)
+    def jc(self, label: str) -> None: self.jcc("jc", label)
+    def jnc(self, label: str) -> None: self.jcc("jnc", label)
+    def jn(self, label: str) -> None: self.jcc("jn", label)
+    def jp(self, label: str) -> None: self.jcc("jp", label)
+    def jv(self, label: str) -> None: self.jcc("jv", label)
+    def jnv(self, label: str) -> None: self.jcc("jnv", label)
+
+    def call(self, label: str) -> None:
+        self.dw(_enc(0xE)); self.daddr(label)
+
+    def halt(self) -> None:
+        self.dw(_enc(0xF))
+
+    def assemble(self) -> List[int]:
+        for offset, label in self.fixups:
+            if label not in self.labels:
+                raise ValueError(f"undefined label: {label}")
+            target = self.labels[label]
+            self.mem[offset] = (target >> 8) & 0xFF
+            self.mem[offset + 1] = target & 0xFF
+        return self.mem
+
+
+# ----------------------------------------------------------------------------
+# Programs
+# ----------------------------------------------------------------------------
+
+ProgramResult = Tuple[List[int], Dict[int, int], int, str]
+
+
+def fib(n: int = 11, mem_size: int = 256) -> ProgramResult:
+    """Iterative Fibonacci F(N), 8-bit wrap.
+
+    F(11) = 89. F(13) = 233 still fits in 8 bits; F(14) = 377 overflows.
+
+    Algorithm: maintain (a, b) = (F(k), F(k+1)); after N steps a = F(N).
+    Per iteration: temp=b, b=a+b, a=temp; n--.
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    a.load(2, "n_addr")           # R2 = n
+    a.load(0, "zero_addr")        # R0 = 0 = F(0)
+    a.load(1, "one_addr")         # R1 = 1 = F(1)
+    a.label("loop")
+    a.load(3, "zero_addr")        # R3 = 0
+    a.cmp(2, 3)                   # n == 0?
+    a.jz("done")
+    a.load(3, "zero_addr")        # R3 = 0
+    a.add(3, 1)                   # R3 = b      (saved old b)
+    a.add(1, 0)                   # R1 = a + b  (new b)
+    a.load(0, "zero_addr")        # R0 = 0
+    a.add(0, 3)                   # R0 = old b  (new a)
+    a.load(3, "one_addr")         # R3 = 1
+    a.sub(2, 3)                   # n--
+    a.jmp("loop")
+    a.label("done")
+    a.store(0, "out")             # OUT = a
+    a.halt()
+
+    a.org(0x80)
+    a.label("zero_addr"); a.db(0)
+    a.label("one_addr");  a.db(1)
+    a.label("n_addr");    a.db(n)
+    a.label("out");       a.db(0)
+
+    mem = a.assemble()
+    expected_a = 0
+    aa, bb = 0, 1
+    for _ in range(n):
+        aa, bb = bb, (aa + bb) & 0xFF
+    expected_a = aa
+    return mem, {a.labels["out"]: expected_a}, 16 * (n + 2), f"Fibonacci F({n}) = {expected_a}"
+
+
+def sum_n(n: int = 10, mem_size: int = 256) -> ProgramResult:
+    """Compute 1 + 2 + ... + N using the Z flag from SUB to terminate.
+
+    No explicit zero register required; SUB sets Z when its result is zero.
+    R0 = accumulator, R1 = counter (n down to 1), R2 = 1.
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    a.load(0, "zero_addr")        # acc = 0
+    a.load(1, "n_addr")           # i = n
+    a.load(2, "one_addr")         # 1
+    a.label("loop")
+    a.add(0, 1)                   # acc += i
+    a.sub(1, 2)                   # i-- (Z set when i becomes 0)
+    a.jnz("loop")
+    a.store(0, "out")
+    a.halt()
+
+    a.org(0x80)
+    a.label("zero_addr"); a.db(0)
+    a.label("one_addr");  a.db(1)
+    a.label("n_addr");    a.db(n)
+    a.label("out");       a.db(0)
+
+    mem = a.assemble()
+    expected = (n * (n + 1) // 2) & 0xFF
+    return mem, {a.labels["out"]: expected}, 4 + 4 * n, f"sum 1..{n} = {expected}"
+
+
+def self_mod_jmp(mem_size: int = 256) -> ProgramResult:
+    """Self-modifying code: writes the JMP target's low byte at runtime.
+
+    Initial JMP target is path_a (writes 0xAA to OUT). The code first
+    overwrites the JMP's address-word LSB so it points to path_b
+    (writes 0xBB). Successful execution lands at path_b, so OUT = 0xBB.
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    a.label("start")
+
+    # Forward-declare the_jmp's LSB address. The first two instructions are
+    # each 4 bytes; the_jmp follows them, so the_jmp = pc + 8 from start.
+    # The JMP's address word is at the_jmp + 2; the LSB byte is at the_jmp + 3.
+    a.labels["jmp_target_lsb"] = a.pc + 8 + 3
+
+    a.load(0, "new_lsb")           # R0 = LSB of path_b address (4 bytes)
+    a.store(0, "jmp_target_lsb")   # patch the JMP's LSB (4 bytes)
+    a.label("the_jmp")
+    a.jmp("path_a")                # initially -> path_a; after patch -> path_b
+    a.label("path_a")
+    a.load(1, "val_a"); a.store(1, "out"); a.halt()
+    a.label("path_b")
+    a.load(1, "val_b"); a.store(1, "out"); a.halt()
+
+    a.org(0x80)
+    a.label("val_a");   a.db(0xAA)
+    a.label("val_b");   a.db(0xBB)
+    # new_lsb_word stores path_b's full 16-bit address; new_lsb labels the LSB byte.
+    a.label("new_lsb_word"); a.daddr("path_b")
+    a.labels["new_lsb"] = a.labels["new_lsb_word"] + 1
+    a.label("out");     a.db(0)
+
+    mem = a.assemble()
+    return mem, {a.labels["out"]: 0xBB}, 30, "self-modifying JMP target"
+
+
+def all_branches(mem_size: int = 256) -> ProgramResult:
+    """Drive all eight conditional jumps; each path writes a unique marker.
+
+    Test plan (each step sets flags, then a Jcc; the branch takes when
+    expected and the corresponding marker is written):
+
+      JZ:  CMP equal -> Z=1 -> taken  -> M[OUT0] = 0xA0
+      JNZ: CMP unequal -> Z=0 -> taken -> M[OUT1] = 0xA1
+      JC:  ADD overflow (255+1) -> C=1 -> taken -> M[OUT2] = 0xA2
+      JNC: ADD no overflow -> C=0 -> taken -> M[OUT3] = 0xA3
+      JN:  SUB result 0xFF (n=1) -> N=1 -> taken -> M[OUT4] = 0xA4
+      JP:  ADD result 1 -> N=0 -> taken -> M[OUT5] = 0xA5
+      JV:  ADD signed overflow (127+1=128) -> V=1 -> taken -> M[OUT6] = 0xA6
+      JNV: ADD no signed overflow (1+1=2) -> V=0 -> taken -> M[OUT7] = 0xA7
+
+    A failure on any branch causes the wrong (or no) marker to be written.
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    # ----- JZ: equal compare -> Z=1 -----
+    a.load(0, "v5"); a.load(1, "v5"); a.cmp(0, 1); a.jz("ok_jz"); a.jmp("fail")
+    a.label("ok_jz"); a.load(2, "m_a0"); a.store(2, "out0")
+
+    # ----- JNZ: unequal compare -> Z=0 -----
+    a.load(0, "v5"); a.load(1, "v3"); a.cmp(0, 1); a.jnz("ok_jnz"); a.jmp("fail")
+    a.label("ok_jnz"); a.load(2, "m_a1"); a.store(2, "out1")
+
+    # ----- JC: 255+1 = 0 with carry -----
+    a.load(0, "v255"); a.load(1, "v1"); a.add(0, 1); a.jc("ok_jc"); a.jmp("fail")
+    a.label("ok_jc"); a.load(2, "m_a2"); a.store(2, "out2")
+
+    # ----- JNC: 1+1 = 2, no carry -----
+    a.load(0, "v1"); a.load(1, "v1"); a.add(0, 1); a.jnc("ok_jnc"); a.jmp("fail")
+    a.label("ok_jnc"); a.load(2, "m_a3"); a.store(2, "out3")
+
+    # ----- JN: 0 - 1 = 0xFF, MSB set -----
+    a.load(0, "v0"); a.load(1, "v1"); a.sub(0, 1); a.jn("ok_jn"); a.jmp("fail")
+    a.label("ok_jn"); a.load(2, "m_a4"); a.store(2, "out4")
+
+    # ----- JP: 0 + 1 = 1, MSB clear -----
+    a.load(0, "v0"); a.load(1, "v1"); a.add(0, 1); a.jp("ok_jp"); a.jmp("fail")
+    a.label("ok_jp"); a.load(2, "m_a5"); a.store(2, "out5")
+
+    # ----- JV: 127 + 1 = 128, signed overflow -----
+    a.load(0, "v127"); a.load(1, "v1"); a.add(0, 1); a.jv("ok_jv"); a.jmp("fail")
+    a.label("ok_jv"); a.load(2, "m_a6"); a.store(2, "out6")
+
+    # ----- JNV: 1 + 1 = 2, no signed overflow -----
+    a.load(0, "v1"); a.load(1, "v1"); a.add(0, 1); a.jnv("ok_jnv"); a.jmp("fail")
+    a.label("ok_jnv"); a.load(2, "m_a7"); a.store(2, "out7")
+
+    a.jmp("end")
+    a.label("fail")
+    a.load(2, "v_fail"); a.store(2, "fail_addr"); a.halt()
+    a.label("end")
+    a.halt()
+
+    # Code runs to ~0xDF; data starts safely after that.
+    a.org(0xE0)
+    a.label("v0");     a.db(0)
+    a.label("v1");     a.db(1)
+    a.label("v3");     a.db(3)
+    a.label("v5");     a.db(5)
+    a.label("v127");   a.db(127)
+    a.label("v255");   a.db(255)
+    a.label("m_a0");   a.db(0xA0)
+    a.label("m_a1");   a.db(0xA1)
+    a.label("m_a2");   a.db(0xA2)
+    a.label("m_a3");   a.db(0xA3)
+    a.label("m_a4");   a.db(0xA4)
+    a.label("m_a5");   a.db(0xA5)
+    a.label("m_a6");   a.db(0xA6)
+    a.label("m_a7");   a.db(0xA7)
+    a.label("v_fail"); a.db(0xEE)
+    a.label("out0");   a.db(0)
+    a.label("out1");   a.db(0)
+    a.label("out2");   a.db(0)
+    a.label("out3");   a.db(0)
+    a.label("out4");   a.db(0)
+    a.label("out5");   a.db(0)
+    a.label("out6");   a.db(0)
+    a.label("out7");   a.db(0)
+    a.label("fail_addr"); a.db(0)
+
+    mem = a.assemble()
+    expected = {
+        a.labels["out0"]: 0xA0,
+        a.labels["out1"]: 0xA1,
+        a.labels["out2"]: 0xA2,
+        a.labels["out3"]: 0xA3,
+        a.labels["out4"]: 0xA4,
+        a.labels["out5"]: 0xA5,
+        a.labels["out6"]: 0xA6,
+        a.labels["out7"]: 0xA7,
+        a.labels["fail_addr"]: 0,    # must remain zero
+    }
+    return mem, expected, 200, "all 8 conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV)"
+
+
+def call_pushes_pc(mem_size: int = 256) -> ProgramResult:
+    """Verify CALL pushes the return address (next-instruction PC) onto the stack.
+
+    SP starts at addr_mask (mem_size - 1). CALL decrements SP and writes the
+    return-address high byte, decrements again and writes the low byte. After
+    HALT we expect:
+      - mem[addr_mask - 2] = low byte of the return address
+      - mem[addr_mask - 1] = high byte of the return address
+      - the no-return code path was NOT taken
+      - the callee was reached
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    a.label("caller")
+    a.load(0, "marker_val")
+    a.store(0, "marker_addr")     # write before CALL
+    a.label("call_site")
+    a.call("callee")
+    # If CALL did not transfer control, this fallthrough store would write 0xDD:
+    a.load(0, "noret_val")
+    a.store(0, "noret_addr")
+    a.halt()
+
+    a.label("callee")
+    a.load(0, "callee_val")
+    a.store(0, "callee_addr")
+    a.halt()
+
+    a.org(0x40)
+    a.label("marker_val");  a.db(0x11)
+    a.label("marker_addr"); a.db(0)
+    a.label("noret_val");   a.db(0xDD)
+    a.label("noret_addr");  a.db(0)
+    a.label("callee_val");  a.db(0x22)
+    a.label("callee_addr"); a.db(0)
+
+    mem = a.assemble()
+    addr_mask = mem_size - 1
+    return_addr = a.labels["call_site"] + 4         # 4-byte CALL instruction
+    expected = {
+        a.labels["marker_addr"]: 0x11,              # pre-CALL store ran
+        a.labels["callee_addr"]: 0x22,              # callee reached
+        a.labels["noret_addr"]: 0,                  # fallthrough did not run
+        (addr_mask - 2) & addr_mask: return_addr & 0xFF,         # ret LSB on stack
+        (addr_mask - 1) & addr_mask: (return_addr >> 8) & 0xFF,  # ret MSB
+    }
+    return mem, expected, 30, "CALL pushes return PC onto stack"
+
+
+def bubble_sort_4(mem_size: int = 256) -> ProgramResult:
+    """Sort a 4-byte array using unrolled compare-swap (3 passes of 3 compares).
+
+    Algorithm: bubble sort, fully unrolled (no inner loops). Each compare-swap
+    is 8 instructions; 3 outer passes x 3 inner positions = 9 swaps -> ~72 instrs.
+
+    For each position i in (0,1,2):
+        if A[i] > A[i+1]:
+            tmp = A[i]; A[i] = A[i+1]; A[i+1] = tmp
+    Repeat 3 times -> sorted ascending.
+    """
+    a = Asm(mem_size)
+
+    addrs = ["a0", "a1", "a2", "a3"]
+
+    a.org(0)
+    for _outer in range(3):
+        for i in range(3):
+            x, y = addrs[i], addrs[i + 1]
+            # Load pair
+            a.load(0, x)            # R0 = A[i]
+            a.load(1, y)            # R1 = A[i+1]
+            a.cmp(0, 1)             # compare A[i] - A[i+1]
+            # If A[i] <= A[i+1] (Z=1 or C=0 from sub-style cmp), skip swap
+            # SUB sets carry when no borrow (a >= b). So:
+            #   a > b  iff  Z=0 and a >= b -> Z=0 and C=1 (the sub didn't borrow)
+            # We want to swap when a > b. JNC (no carry / borrow) means a < b -> skip swap.
+            # If a == b (Z=1) we also skip. So: jump-skip when JZ OR JNC.
+            # Easier: compute (a > b) by checking C=1 AND Z=0. Use JZ to skip on equal,
+            # then JNC to skip on a < b. Otherwise fall through to swap.
+            skip_lbl = f"skip_{_outer}_{i}"
+            a.jz(skip_lbl)          # equal -> skip
+            a.jnc(skip_lbl)         # a < b (sub borrowed) -> skip
+            # swap: store R0 -> y, R1 -> x
+            a.store(1, x)
+            a.store(0, y)
+            a.label(skip_lbl)
+    a.halt()
+
+    # Initial unsorted array; code runs to ~0xEC, so data starts at 0xF0.
+    initial = [42, 7, 200, 19]
+    a.org(0xF0)
+    for name, val in zip(addrs, initial):
+        a.label(name); a.db(val)
+
+    mem = a.assemble()
+    sorted_vals = sorted(initial)
+    expected = {a.labels[name]: v for name, v in zip(addrs, sorted_vals)}
+    return mem, expected, 800, f"bubble sort {initial} -> {sorted_vals}"
+
+
+def cross_check_mul(mem_size: int = 256) -> ProgramResult:
+    """Cross-check the threshold MUL circuit against repeated ADD.
+
+    Computes A * B two ways and stores both results:
+      repeated add: acc = 0; loop B times: acc += A  -> OUT_ADD
+      direct:       R0 = A; MUL R0, R1 (R1 = B)      -> OUT_MUL
+    The test verifies the two agree with the expected product.
+    """
+    a = Asm(mem_size)
+    A_VAL = 17
+    B_VAL = 9
+    expected_product = (A_VAL * B_VAL) & 0xFF
+
+    a.org(0)
+    # --- direct multiply ---
+    a.load(0, "A")
+    a.load(1, "B")
+    a.mul(0, 1)
+    a.store(0, "out_mul")
+    # --- repeated-add multiply ---
+    a.load(0, "zero")            # acc = 0
+    a.load(1, "A")               # addend
+    a.load(2, "B")               # counter
+    a.load(3, "one")             # 1
+    a.label("rep_loop")
+    a.add(0, 1)                  # acc += A
+    a.sub(2, 3)                  # B--
+    a.jnz("rep_loop")
+    a.store(0, "out_add")
+    a.halt()
+
+    a.org(0x80)
+    a.label("A");        a.db(A_VAL)
+    a.label("B");        a.db(B_VAL)
+    a.label("zero");     a.db(0)
+    a.label("one");      a.db(1)
+    a.label("out_mul");  a.db(0)
+    a.label("out_add");  a.db(0)
+
+    mem = a.assemble()
+    expected = {
+        a.labels["out_mul"]: expected_product,
+        a.labels["out_add"]: expected_product,
+    }
+    return mem, expected, 80, f"MUL vs repeated ADD: {A_VAL} * {B_VAL} = {expected_product}"
+
+
+def div_via_repeated_sub(mem_size: int = 256) -> ProgramResult:
+    """Compute floor(A/B) and (A mod B) by repeated subtraction.
+
+    Loop: while A >= B { A -= B; quotient += 1 }
+    Uses CMP + JC (carry-set on no-borrow), SUB, ADD, JMP, STORE, HALT.
+
+    Cross-checked against the on-chip 8-bit DIV opcode (0x8) via a
+    second pass that uses DIV directly. Both quotients written to OUT
+    locations; the test verifies they match.
+    """
+    A_VAL = 100
+    B_VAL = 7
+    expected_q = A_VAL // B_VAL          # 14
+    expected_r = A_VAL % B_VAL           # 2
+
+    a = Asm(mem_size)
+
+    a.org(0)
+    # ---- Repeated-subtraction division ----
+    a.load(0, "A")               # R0 = A (will become remainder)
+    a.load(1, "B")               # R1 = B (divisor)
+    a.load(2, "ZERO")            # R2 = 0 (will become quotient)
+    a.load(3, "ONE")             # R3 = 1 (increment)
+
+    a.label("loop")
+    a.cmp(0, 1)                  # CMP R0, R1; carry=1 (no-borrow) iff R0 >= R1
+    a.jnc("done")                # if R0 < R1 (carry=0), exit loop
+    a.sub(0, 1)                  # R0 -= B
+    a.add(2, 3)                  # quotient += 1
+    a.jmp("loop")
+
+    a.label("done")
+    a.store(2, "OUT_Q_RPT")      # quotient via repeated sub
+    a.store(0, "OUT_R_RPT")      # remainder via repeated sub
+
+    # ---- Direct DIV opcode for cross-check ----
+    a.load(0, "A")
+    a.load(1, "B")
+    a.dw(_enc(0x8, 0, 1, 0))     # DIV R0, R1   -> R0 = R0 / R1 (8-bit DIV)
+    a.store(0, "OUT_Q_DIV")
+    a.halt()
+
+    a.org(0x80)
+    a.label("A");          a.db(A_VAL)
+    a.label("B");          a.db(B_VAL)
+    a.label("ZERO");       a.db(0)
+    a.label("ONE");        a.db(1)
+    a.label("OUT_Q_RPT");  a.db(0)
+    a.label("OUT_R_RPT");  a.db(0)
+    a.label("OUT_Q_DIV");  a.db(0)
+
+    mem = a.assemble()
+    expected = {
+        a.labels["OUT_Q_RPT"]: expected_q,
+        a.labels["OUT_R_RPT"]: expected_r,
+        a.labels["OUT_Q_DIV"]: expected_q,
+    }
+    return mem, expected, 4 * (A_VAL // B_VAL + 4) + 12, (
+        f"{A_VAL} / {B_VAL}: quotient {expected_q} (repeated SUB) "
+        f"matches DIV opcode result; remainder {expected_r}"
+    )
+
+
+def bitwise_chain(mem_size: int = 256) -> ProgramResult:
+    """Run a chain of bitwise ops and verify each intermediate value.
+
+    Sequence:
+        R0 = A & B           (AND)
+        R0 = R0 | C           (OR)
+        R0 = R0 ^ D           (XOR)
+        R0 = R0 << 1          (SHL)
+        R0 = R0 >> 1          (SHR)
+    Stores R0 after each step. Verifies all intermediate values to
+    catch any single-op regression.
+    """
+    A = 0xCC  # 11001100
+    B = 0xF0  # 11110000
+    C = 0x0F  # 00001111
+    D = 0xAA  # 10101010
+
+    s1 = A & B               # 0xC0
+    s2 = s1 | C              # 0xCF
+    s3 = s2 ^ D              # 0x65
+    s4 = (s3 << 1) & 0xFF    # 0xCA
+    s5 = s4 >> 1             # 0x65
+
+    a = Asm(mem_size)
+    a.org(0)
+    a.load(0, "A"); a.load(1, "B"); a.and_(0, 1); a.store(0, "S1")
+    a.load(1, "C"); a.or_(0, 1);                     a.store(0, "S2")
+    a.load(1, "D"); a.xor(0, 1);                    a.store(0, "S3")
+    a.shl(0);                                        a.store(0, "S4")
+    a.shr(0);                                        a.store(0, "S5")
+    a.halt()
+
+    a.org(0x80)
+    a.label("A"); a.db(A)
+    a.label("B"); a.db(B)
+    a.label("C"); a.db(C)
+    a.label("D"); a.db(D)
+    a.label("S1"); a.db(0)
+    a.label("S2"); a.db(0)
+    a.label("S3"); a.db(0)
+    a.label("S4"); a.db(0)
+    a.label("S5"); a.db(0)
+
+    mem = a.assemble()
+    expected = {
+        a.labels["S1"]: s1,
+        a.labels["S2"]: s2,
+        a.labels["S3"]: s3,
+        a.labels["S4"]: s4,
+        a.labels["S5"]: s5,
+    }
+    return mem, expected, 30, (
+        f"bitwise chain AND/OR/XOR/SHL/SHR -> {s1:#x},{s2:#x},{s3:#x},{s4:#x},{s5:#x}"
+    )
+
+
+def flags_policy(mem_size: int = 256) -> ProgramResult:
+    """Pin the architectural flag policy: only ADD, SUB, MUL, and CMP write
+    FLAGS; bitwise ops, shifts, DIV, LOAD, and STORE leave them unchanged.
+
+    Three probes, each setting flags with an arithmetic op, disturbing the
+    machine with a non-flag-writing op, then branching on the original flags:
+
+      1. CMP equal -> Z=1; AND producing a nonzero result; JZ must still take.
+      2. CMP unequal -> Z=0; XOR producing a zero result; JZ must NOT take.
+      3. ADD 255+1 -> C=1; SHR (shifts a 0 out); JC must still take.
+
+    Any runtime that lets the intervening op rewrite FLAGS lands on the
+    wrong path and writes the 0xEE failure marker instead.
+    """
+    a = Asm(mem_size)
+
+    a.org(0)
+    # ----- Probe 1: Z=1 survives AND (nonzero result) -----
+    a.load(0, "v5"); a.load(1, "v5"); a.cmp(0, 1)      # Z=1
+    a.load(2, "v3"); a.load(3, "v1"); a.and_(2, 3)     # R2 = 3 & 1 = 1
+    a.jz("p1_ok")
+    a.load(2, "m_fail"); a.store(2, "out1"); a.jmp("p2")
+    a.label("p1_ok")
+    a.load(2, "m_b1"); a.store(2, "out1")
+
+    # ----- Probe 2: Z=0 survives XOR (zero result) -----
+    a.label("p2")
+    a.load(0, "v5"); a.load(1, "v3"); a.cmp(0, 1)      # Z=0
+    a.xor(2, 2)                                        # R2 = 0
+    a.jz("p2_fail")
+    a.load(2, "m_b2"); a.store(2, "out2"); a.jmp("p3")
+    a.label("p2_fail")
+    a.load(2, "m_fail"); a.store(2, "out2")
+
+    # ----- Probe 3: C=1 survives SHR -----
+    a.label("p3")
+    a.load(0, "v255"); a.load(1, "v1"); a.add(0, 1)    # C=1
+    a.load(2, "v3"); a.shr(2)                          # R2 = 1
+    a.jc("p3_ok")
+    a.load(2, "m_fail"); a.store(2, "out3"); a.jmp("done")
+    a.label("p3_ok")
+    a.load(2, "m_b3"); a.store(2, "out3")
+    a.label("done")
+    a.halt()
+
+    a.org(0xA0)
+    a.label("v1");     a.db(1)
+    a.label("v3");     a.db(3)
+    a.label("v5");     a.db(5)
+    a.label("v255");   a.db(255)
+    a.label("m_b1");   a.db(0xB1)
+    a.label("m_b2");   a.db(0xB2)
+    a.label("m_b3");   a.db(0xB3)
+    a.label("m_fail"); a.db(0xEE)
+    a.label("out1");   a.db(0)
+    a.label("out2");   a.db(0)
+    a.label("out3");   a.db(0)
+
+    mem = a.assemble()
+    expected = {
+        a.labels["out1"]: 0xB1,
+        a.labels["out2"]: 0xB2,
+        a.labels["out3"]: 0xB3,
+    }
+    return mem, expected, 60, "flag policy: AND/XOR/SHR leave FLAGS unchanged"
+
+
+SUITE = [
+    ("fib", lambda mem_size: fib(11, mem_size)),
+    ("sum_n", lambda mem_size: sum_n(10, mem_size)),
+    ("self_mod_jmp", lambda mem_size: self_mod_jmp(mem_size)),
+    ("all_branches", lambda mem_size: all_branches(mem_size)),
+    ("call_pushes_pc", lambda mem_size: call_pushes_pc(mem_size)),
+    ("bubble_sort_4", lambda mem_size: bubble_sort_4(mem_size)),
+    ("cross_check_mul", lambda mem_size: cross_check_mul(mem_size)),
+    ("div_via_repeated_sub", lambda mem_size: div_via_repeated_sub(mem_size)),
+    ("bitwise_chain", lambda mem_size: bitwise_chain(mem_size)),
+    ("flags_policy", lambda mem_size: flags_policy(mem_size)),
+]
+

eval_all.py

@@ -0,0 +1,850 @@
+"""
+Unified evaluation harness for any threshold-computer variant.
+
+Drops the `--cpu-test` smoke test (which was hardcoded to 16-bit/64KB) and
+adds variant-aware sweep modes. The same harness handles every (data_bits,
+addr_bits) configuration: it reads the manifest from each safetensors file,
+runs the BatchedFitnessEvaluator at the right device, and reports per-file
+plus per-category results.
+
+Usage:
+    python eval_all.py path/to/file.safetensors          # one file
+    python eval_all.py variants/                          # every .safetensors in dir
+    python eval_all.py --device cpu variants/             # CPU only (default)
+    python eval_all.py --pop_size 32 variants/            # batched pop eval
+    python eval_all.py --debug path/to/file.safetensors   # per-circuit detail
+    python eval_all.py --cpu-program PATH                 # also run an assembled program
+                                                          # through the threshold CPU
+                                                          # sized to the file's manifest
+
+Exit code:
+    0 if all files PASS (fitness >= 0.9999)
+    N where N is the number of FAILing files
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+import torch
+from safetensors import safe_open
+
+# Reuse eval.py's evaluator (variant-aware)
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from eval import (
+    BatchedFitnessEvaluator,
+    create_population,
+    load_model,
+    get_manifest,
+    heaviside,
+    int_to_bits,
+    bits_to_int,
+    bits_msb_to_lsb,
+)
+
+
+# ---------------------------------------------------------------------------
+# Variant-aware threshold ALU + CPU
+# ---------------------------------------------------------------------------
+
+class GenericThresholdALU:
+    """Variant-aware threshold ALU walking the gates of a loaded variant.
+
+    8-bit primitives: add8/sub8/mul8/div8, packed bitwise and8/or8/xor8,
+    shl8/shr8, and the bit-cascade comparators via cmp8. Width-generic
+    add_n/sub_n/mul_n/cmp_n cover the 16- and 32-bit circuit families.
+    """
+
+    def __init__(self, tensors: Dict[str, torch.Tensor], data_bits: int):
+        self.T = tensors
+        self.data_bits = data_bits
+
+    def _g(self, name, inputs):
+        w = self.T[name + ".weight"].view(-1)
+        b = self.T[name + ".bias"].view(-1)
+        return int(heaviside((torch.tensor(inputs, dtype=torch.float32) * w).sum() + b).item())
+
+    def _xor_or_nand(self, prefix, inputs):
+        a, b_ = inputs
+        h_or = self._g(f"{prefix}.layer1.or", [a, b_])
+        h_nand = self._g(f"{prefix}.layer1.nand", [a, b_])
+        return self._g(f"{prefix}.layer2", [h_or, h_nand])
+
+    def _fa(self, prefix, a, b, cin):
+        s1 = self._xor_or_nand(f"{prefix}.ha1.sum", [a, b])
+        c1 = self._g(f"{prefix}.ha1.carry", [a, b])
+        s2 = self._xor_or_nand(f"{prefix}.ha2.sum", [s1, cin])
+        c2 = self._g(f"{prefix}.ha2.carry", [s1, cin])
+        cout = self._g(f"{prefix}.carry_or", [c1, c2])
+        return s2, cout
+
+    def add8(self, a, b):
+        a_lsb = list(reversed(int_to_bits(a, 8)))
+        b_lsb = list(reversed(int_to_bits(b, 8)))
+        carry = 0
+        s_lsb = []
+        for i in range(8):
+            s, carry = self._fa(f"arithmetic.ripplecarry8bit.fa{i}", a_lsb[i], b_lsb[i], carry)
+            s_lsb.append(s)
+        return bits_to_int(list(reversed(s_lsb))), carry
+
+    def sub8(self, a, b):
+        a_lsb = list(reversed(int_to_bits(a, 8)))
+        b_lsb = list(reversed(int_to_bits(b, 8)))
+        carry = 1
+        d_lsb = []
+        for i in range(8):
+            notb = self._g(f"arithmetic.sub8bit.notb{i}", [b_lsb[i]])
+            x1 = self._xor_or_nand(f"arithmetic.sub8bit.fa{i}.xor1", [a_lsb[i], notb])
+            x2 = self._xor_or_nand(f"arithmetic.sub8bit.fa{i}.xor2", [x1, carry])
+            and1 = self._g(f"arithmetic.sub8bit.fa{i}.and1", [a_lsb[i], notb])
+            and2 = self._g(f"arithmetic.sub8bit.fa{i}.and2", [x1, carry])
+            carry = self._g(f"arithmetic.sub8bit.fa{i}.or_carry", [and1, and2])
+            d_lsb.append(x2)
+        return bits_to_int(list(reversed(d_lsb))), carry
+
+    _CMP_KIND = {"greaterthan": "gt", "lessthan": "lt", "eq": "eq", "equality": "eq",
+                 "greaterorequal": "ge", "lessorequal": "le"}
+
+    def _cmp_bit_cascade(self, cmp_prefix: str, finals: Dict[str, str],
+                         a_bits_msb: List[int], b_bits_msb: List[int], kind: str) -> int:
+        """Walk an add_bit_cascade_compare structure (bit 0 is the MSB)."""
+        bits = len(a_bits_msb)
+        bit_gt, bit_lt, bit_eq = [], [], []
+        for i in range(bits):
+            ab = [a_bits_msb[i], b_bits_msb[i]]
+            bit_gt.append(self._g(f"{cmp_prefix}.bit{i}.gt", ab))
+            bit_lt.append(self._g(f"{cmp_prefix}.bit{i}.lt", ab))
+            eq_and = self._g(f"{cmp_prefix}.bit{i}.eq.layer1.and", ab)
+            eq_nor = self._g(f"{cmp_prefix}.bit{i}.eq.layer1.nor", ab)
+            bit_eq.append(self._g(f"{cmp_prefix}.bit{i}.eq", [eq_and, eq_nor]))
+        cas_gt, cas_lt = [bit_gt[0]], [bit_lt[0]]
+        for i in range(1, bits):
+            eq_pref = self._g(f"{cmp_prefix}.cascade.eq_prefix.bit{i}", bit_eq[:i])
+            cas_gt.append(self._g(f"{cmp_prefix}.cascade.gt.bit{i}", [eq_pref, bit_gt[i]]))
+            cas_lt.append(self._g(f"{cmp_prefix}.cascade.lt.bit{i}", [eq_pref, bit_lt[i]]))
+        if kind == "gt":
+            return self._g(finals["gt"], cas_gt)
+        if kind == "lt":
+            return self._g(finals["lt"], cas_lt)
+        if kind == "eq":
+            return self._g(finals["eq"], bit_eq)
+        if kind == "ge":
+            lt = self._g(finals["lt"], cas_lt)
+            not_lt = self._g(finals["ge"] + ".not_lt", [lt])
+            return self._g(finals["ge"], [not_lt])
+        if kind == "le":
+            gt = self._g(finals["gt"], cas_gt)
+            not_gt = self._g(finals["le"] + ".not_gt", [gt])
+            return self._g(finals["le"], [not_gt])
+        raise ValueError(kind)
+
+    def cmp8(self, a, b, kind):
+        return self.cmp_n(a, b, kind, 8)
+
+    def mul8(self, a, b):
+        ab = int_to_bits(a, 8)
+        bb = int_to_bits(b, 8)
+        result = 0
+        for j in range(8):
+            if bb[j] == 0:
+                continue
+            row = 0
+            for i in range(8):
+                pp = self._g(f"alu.alu8bit.mul.pp.a{i}b{j}", [ab[i], bb[j]])
+                row |= (pp << (7 - i))
+            shift = 7 - j
+            result, _ = self.add8(result & 0xFF, (row << shift) & 0xFF)
+        return result & 0xFF
+
+    def _packed_pair_op(self, name: str, a: int, b: int) -> int:
+        """8 parallel 2-input gates stored packed: weight [16], bias [8]."""
+        a_bits = int_to_bits(a, 8)
+        b_bits = int_to_bits(b, 8)
+        w = self.T[f"{name}.weight"].view(-1)
+        bias = self.T[f"{name}.bias"].view(-1)
+        out = []
+        for bit in range(8):
+            inp = torch.tensor([float(a_bits[bit]), float(b_bits[bit])])
+            out.append(int(heaviside((inp * w[bit * 2:bit * 2 + 2]).sum() + bias[bit]).item()))
+        return bits_to_int(out)
+
+    def and8(self, a, b):
+        return self._packed_pair_op("alu.alu8bit.and", a, b)
+
+    def or8(self, a, b):
+        return self._packed_pair_op("alu.alu8bit.or", a, b)
+
+    def xor8(self, a, b):
+        a_bits = int_to_bits(a, 8)
+        b_bits = int_to_bits(b, 8)
+        w_or = self.T["alu.alu8bit.xor.layer1.or.weight"].view(-1)
+        b_or = self.T["alu.alu8bit.xor.layer1.or.bias"].view(-1)
+        w_nand = self.T["alu.alu8bit.xor.layer1.nand.weight"].view(-1)
+        b_nand = self.T["alu.alu8bit.xor.layer1.nand.bias"].view(-1)
+        w2 = self.T["alu.alu8bit.xor.layer2.weight"].view(-1)
+        b2 = self.T["alu.alu8bit.xor.layer2.bias"].view(-1)
+        out = []
+        for bit in range(8):
+            inp = torch.tensor([float(a_bits[bit]), float(b_bits[bit])])
+            h_or = heaviside((inp * w_or[bit * 2:bit * 2 + 2]).sum() + b_or[bit])
+            h_nand = heaviside((inp * w_nand[bit * 2:bit * 2 + 2]).sum() + b_nand[bit])
+            hidden = torch.stack([h_or, h_nand])
+            out.append(int(heaviside((hidden * w2[bit * 2:bit * 2 + 2]).sum() + b2[bit]).item()))
+        return bits_to_int(out)
+
+    def shl8(self, a):
+        a_bits = int_to_bits(a, 8)
+        out = []
+        for bit in range(8):
+            src = float(a_bits[bit + 1]) if bit < 7 else 0.0
+            out.append(self._g(f"alu.alu8bit.shl.bit{bit}", [src]))
+        return bits_to_int(out)
+
+    def shr8(self, a):
+        a_bits = int_to_bits(a, 8)
+        out = []
+        for bit in range(8):
+            src = float(a_bits[bit - 1]) if bit > 0 else 0.0
+            out.append(self._g(f"alu.alu8bit.shr.bit{bit}", [src]))
+        return bits_to_int(out)
+
+    def div8(self, a, b):
+        """8-bit restoring division through the per-stage bit-cascade GE
+        comparators and the sub8 subtractor. Returns (quotient, remainder);
+        divide-by-zero yields (0xFF, a)."""
+        if b == 0:
+            return 0xFF, a
+        a_bits = int_to_bits(a, 8)
+        div_bits = int_to_bits(b, 8)
+        quotient = 0
+        remainder = 0
+        for stage in range(8):
+            remainder = ((remainder << 1) | a_bits[stage]) & 0xFF
+            prefix = f"alu.alu8bit.div.stage{stage}"
+            finals = {
+                "gt": f"{prefix}.cmp_bc.gt",
+                "lt": f"{prefix}.cmp_bc.lt",
+                "eq": f"{prefix}.cmp_bc.eq",
+                "ge": f"{prefix}.cmp",
+                "le": f"{prefix}.cmp_bc.le",
+            }
+            ge = self._cmp_bit_cascade(f"{prefix}.cmp_bc", finals,
+                                       int_to_bits(remainder, 8), div_bits, "ge")
+            if ge:
+                remainder, _ = self.sub8(remainder, b)
+                quotient = (quotient << 1) | 1
+            else:
+                quotient = quotient << 1
+        return quotient & 0xFF, remainder & 0xFF
+
+    # ----- N-bit primitives (for 16-bit and 32-bit variants) ----------------
+
+    def add_n(self, a: int, b: int, bits: int):
+        """Width-generic ripple-carry add via arithmetic.ripplecarry{N}bit."""
+        prefix = f"arithmetic.ripplecarry{bits}bit"
+        a_lsb = list(reversed(int_to_bits(a, bits)))
+        b_lsb = list(reversed(int_to_bits(b, bits)))
+        carry = 0
+        s_lsb = []
+        for i in range(bits):
+            s, carry = self._fa(f"{prefix}.fa{i}", a_lsb[i], b_lsb[i], carry)
+            s_lsb.append(s)
+        return bits_to_int(list(reversed(s_lsb))), carry
+
+    def sub_n(self, a: int, b: int, bits: int):
+        """N-bit two's-complement subtract via arithmetic.sub{N}bit (N >= 16).
+
+        Structure (per build.add_sub_nbits): N NOT gates + N standard full adders.
+        """
+        prefix = f"arithmetic.sub{bits}bit"
+        a_lsb = list(reversed(int_to_bits(a, bits)))
+        b_lsb = list(reversed(int_to_bits(b, bits)))
+        # NOT each B bit
+        notb = [self._g(f"{prefix}.not_b.bit{i}", [b_lsb[i]]) for i in range(bits)]
+        carry = 1  # carry-in = 1 for two's-complement
+        d_lsb = []
+        for i in range(bits):
+            s, carry = self._fa(f"{prefix}.fa{i}", a_lsb[i], notb[i], carry)
+            d_lsb.append(s)
+        return bits_to_int(list(reversed(d_lsb))), carry
+
+    def cmp_n(self, a: int, b: int, kind: str, bits: int):
+        """N-bit unsigned comparison via the bit-cascade comparator family
+        (arithmetic.cmp{N}bit.* per build.add_bit_cascade_compare)."""
+        short = self._CMP_KIND[kind]
+        finals = {
+            "gt": f"arithmetic.greaterthan{bits}bit",
+            "lt": f"arithmetic.lessthan{bits}bit",
+            "eq": f"arithmetic.equality{bits}bit",
+            "ge": f"arithmetic.greaterorequal{bits}bit",
+            "le": f"arithmetic.lessorequal{bits}bit",
+        }
+        return self._cmp_bit_cascade(f"arithmetic.cmp{bits}bit", finals,
+                                     int_to_bits(a, bits), int_to_bits(b, bits), short)
+
+    def mul_n(self, a: int, b: int, bits: int):
+        """N-bit shift-add multiply (low N bits only)."""
+        ab = int_to_bits(a, bits)
+        bb = int_to_bits(b, bits)
+        mask = (1 << bits) - 1
+        result = 0
+        for j in range(bits):
+            if bb[j] == 0:
+                continue
+            row = 0
+            for i in range(bits):
+                pp = self._g(f"alu.alu{bits}bit.mul.pp.a{i}b{j}", [ab[i], bb[j]])
+                row |= (pp << (bits - 1 - i))
+            shift = (bits - 1) - j
+            result, _ = self.add_n(result & mask, (row << shift) & mask, bits)
+        return result & mask
+
+
+class GenericThresholdCPU:
+    """Variant-aware CPU runtime. Sized from the variant's manifest."""
+
+    def __init__(self, tensors: Dict[str, torch.Tensor]):
+        self.T = tensors
+        m = get_manifest(tensors)
+        self.data_bits = m["data_bits"]
+        self.addr_bits = m["addr_bits"]
+        self.mem_bytes = m["memory_bytes"]
+        # 8-bit CPU primitives (ripplecarry8bit, sub8bit, alu.alu8bit.*, memory.*,
+        # control.*) are present in every variant regardless of manifest data_bits.
+        # Wider data widths simply add additional standalone ALU primitives.
+        if self.mem_bytes == 0:
+            raise NotImplementedError(
+                "Pure-ALU variants have no memory; cannot run CPU programs"
+            )
+        self.alu = GenericThresholdALU(tensors, 8)
+
+    def _addr_decode(self, addr):
+        bits = torch.tensor(int_to_bits(addr, self.addr_bits), dtype=torch.float32)
+        w = self.T["memory.addr_decode.weight"]
+        b = self.T["memory.addr_decode.bias"]
+        return heaviside((w * bits).sum(dim=1) + b)
+
+    def mem_read(self, mem, addr):
+        sel = self._addr_decode(addr)
+        mem_bits = torch.tensor(
+            [int_to_bits(byte, 8) for byte in mem], dtype=torch.float32
+        )
+        and_w = self.T["memory.read.and.weight"]
+        and_b = self.T["memory.read.and.bias"]
+        or_w = self.T["memory.read.or.weight"]
+        or_b = self.T["memory.read.or.bias"]
+        out = []
+        for bit in range(8):
+            inp = torch.stack([mem_bits[:, bit], sel], dim=1)
+            and_out = heaviside((inp * and_w[bit]).sum(dim=1) + and_b[bit])
+            out.append(int(heaviside((and_out * or_w[bit]).sum() + or_b[bit]).item()))
+        return bits_to_int(out)
+
+    def mem_write(self, mem, addr, value):
+        sel = self._addr_decode(addr)
+        data_bits = torch.tensor(int_to_bits(value, 8), dtype=torch.float32)
+        mem_bits = torch.tensor(
+            [int_to_bits(byte, 8) for byte in mem], dtype=torch.float32
+        )
+        sel_w = self.T["memory.write.sel.weight"]
+        sel_b = self.T["memory.write.sel.bias"]
+        nsel_w = self.T["memory.write.nsel.weight"].squeeze(1)
+        nsel_b = self.T["memory.write.nsel.bias"]
+        and_old_w = self.T["memory.write.and_old.weight"]
+        and_old_b = self.T["memory.write.and_old.bias"]
+        and_new_w = self.T["memory.write.and_new.weight"]
+        and_new_b = self.T["memory.write.and_new.bias"]
+        or_w = self.T["memory.write.or.weight"]
+        or_b = self.T["memory.write.or.bias"]
+        we = torch.ones_like(sel)
+        sel_inp = torch.stack([sel, we], dim=1)
+        write_sel = heaviside((sel_inp * sel_w).sum(dim=1) + sel_b)
+        nsel = heaviside(write_sel * nsel_w + nsel_b)
+        for bit in range(8):
+            old = mem_bits[:, bit]
+            data_bit = data_bits[bit].expand(self.mem_bytes)
+            inp_old = torch.stack([old, nsel], dim=1)
+            inp_new = torch.stack([data_bit, write_sel], dim=1)
+            and_old = heaviside((inp_old * and_old_w[:, bit]).sum(dim=1) + and_old_b[:, bit])
+            and_new = heaviside((inp_new * and_new_w[:, bit]).sum(dim=1) + and_new_b[:, bit])
+            or_inp = torch.stack([and_old, and_new], dim=1)
+            new_bit = heaviside((or_inp * or_w[:, bit]).sum(dim=1) + or_b[:, bit])
+            mem_bits[:, bit] = new_bit
+        return [bits_to_int([int(b) for b in mem_bits[i].tolist()]) for i in range(self.mem_bytes)]
+
+    # Conditions 0..7 map to (mux circuit, flag index into [Z, N, C, V]).
+    # Odd conditions are the negated forms; the mux select is the complemented
+    # flag (the circuits are plain per-bit muxes -- see control.* .inputs,
+    # which name the select $not_zero / $not_carry / ... for the odd forms).
+    _JCC = [("control.jz", 0), ("control.jnz", 0), ("control.jc", 2), ("control.jnc", 2),
+            ("control.jn", 1), ("control.jp", 1), ("control.jv", 3), ("control.jnv", 3)]
+
+    def _jcc_pc(self, circuit: str, pc: int, target: int, sel: int) -> int:
+        """Per-bit 2:1 mux over the PC: sel ? target : pc, addr_bits wide."""
+        pc_bits = int_to_bits(pc, self.addr_bits)
+        t_bits = int_to_bits(target, self.addr_bits)
+        out = []
+        for bit in range(self.addr_bits):
+            bp = f"{circuit}.bit{bit}"
+            not_sel = self.alu._g(f"{bp}.not_sel", [float(sel)])
+            and_a = self.alu._g(f"{bp}.and_a", [float(pc_bits[bit]), not_sel])
+            and_b = self.alu._g(f"{bp}.and_b", [float(t_bits[bit]), float(sel)])
+            out.append(self.alu._g(f"{bp}.or", [and_a, and_b]))
+        return bits_to_int(out)
+
+    def _sp_dec(self, sp: int) -> int:
+        """SP - 1 through the control.push.sp_dec borrow chain. Gate bit index
+        is MSB-first (bit addr_bits-1 is the LSB); the bit complement feeding
+        each borrow AND is fixed wiring, as in the gate fitness suite."""
+        bits = int_to_bits(sp, self.addr_bits)
+        out = [0] * self.addr_bits
+        borrow = 1
+        for bit in range(self.addr_bits - 1, -1, -1):
+            bp = f"control.push.sp_dec.bit{bit}"
+            h_or = self.alu._g(f"{bp}.xor.layer1.or", [float(bits[bit]), float(borrow)])
+            h_nand = self.alu._g(f"{bp}.xor.layer1.nand", [float(bits[bit]), float(borrow)])
+            out[bit] = self.alu._g(f"{bp}.xor.layer2", [h_or, h_nand])
+            borrow = self.alu._g(f"{bp}.borrow", [float(1 - bits[bit]), float(borrow)])
+        return bits_to_int(out)
+
+    def step(self, state):
+        if state["halted"]:
+            return state
+        s = dict(state)
+        s["mem"] = state["mem"][:]
+        s["regs"] = state["regs"][:]
+        s["flags"] = state["flags"][:]
+        addr_mask = (1 << self.addr_bits) - 1
+        pc = s["pc"]
+        hi = self.mem_read(s["mem"], pc & addr_mask)
+        lo = self.mem_read(s["mem"], (pc + 1) & addr_mask)
+        ir = ((hi & 0xFF) << 8) | (lo & 0xFF)
+        opcode = (ir >> 12) & 0xF
+        rd = (ir >> 10) & 0x3
+        rs = (ir >> 8) & 0x3
+        imm = ir & 0xFF
+        next_pc = (pc + 2) & addr_mask
+        addr_full = None
+        if opcode in (0xA, 0xB, 0xC, 0xD, 0xE):
+            ah = self.mem_read(s["mem"], next_pc)
+            al = self.mem_read(s["mem"], (next_pc + 1) & addr_mask)
+            addr_full = ((ah & 0xFF) << 8) | (al & 0xFF)
+            next_pc = (next_pc + 2) & addr_mask
+        addr = (addr_full & addr_mask) if addr_full is not None else None
+        a = s["regs"][rd]
+        b = s["regs"][rs]
+        result = a
+        carry = 0
+        overflow = 0
+        write_result = True
+        if opcode == 0x0:
+            result, carry = self.alu.add8(a, b)
+            overflow = 1 if (((a ^ result) & (b ^ result)) & 0x80) else 0
+        elif opcode == 0x1:
+            result, carry = self.alu.sub8(a, b)
+            overflow = 1 if (((a ^ b) & (a ^ result)) & 0x80) else 0
+        elif opcode == 0x2:  # AND
+            result = self.alu.and8(a, b)
+        elif opcode == 0x3:  # OR
+            result = self.alu.or8(a, b)
+        elif opcode == 0x4:  # XOR
+            result = self.alu.xor8(a, b)
+        elif opcode == 0x5:  # SHL by 1 (8-bit)
+            result = self.alu.shl8(a)
+        elif opcode == 0x6:  # SHR by 1
+            result = self.alu.shr8(a)
+        elif opcode == 0x7:
+            result = self.alu.mul8(a, b)
+        elif opcode == 0x8:  # DIV (sets R[d] = R[d] / R[s]; 0xFF on divide by zero)
+            result, _ = self.alu.div8(a, b)
+        elif opcode == 0x9:
+            r2, carry = self.alu.sub8(a, b)
+            z = 1 if r2 == 0 else 0
+            n = 1 if (r2 & 0x80) else 0
+            v = 1 if (((a ^ b) & (a ^ r2)) & 0x80) else 0
+            s["flags"] = [z, n, carry, v]
+            write_result = False
+        elif opcode == 0xA:
+            result = self.mem_read(s["mem"], addr)
+        elif opcode == 0xB:
+            s["mem"] = self.mem_write(s["mem"], addr, b & 0xFF)
+            write_result = False
+        elif opcode == 0xC:
+            s["pc"] = addr
+            return s
+        elif opcode == 0xD:
+            cond = imm & 0x7
+            circuit, flag_idx = self._JCC[cond]
+            flag = s["flags"][flag_idx]
+            sel = flag if cond % 2 == 0 else 1 - flag
+            s["pc"] = self._jcc_pc(circuit, next_pc, addr, sel)
+            return s
+        elif opcode == 0xE:  # CALL: push return address (next_pc), set PC = addr
+            ret_addr = next_pc & 0xFFFF
+            sp = s.get("sp", addr_mask)
+            sp = self._sp_dec(sp)
+            s["mem"] = self.mem_write(s["mem"], sp, (ret_addr >> 8) & 0xFF)
+            sp = self._sp_dec(sp)
+            s["mem"] = self.mem_write(s["mem"], sp, ret_addr & 0xFF)
+            s["sp"] = sp
+            s["pc"] = addr
+            return s
+        elif opcode == 0xF:
+            s["halted"] = True
+            return s
+
+        if write_result and opcode != 0x9:
+            s["regs"][rd] = result & 0xFF
+        # Flag policy: ADD, SUB, MUL (and CMP, handled in its branch) write
+        # Z/N/C/V; MUL clears C and V. All other opcodes leave FLAGS unchanged.
+        if opcode in (0x0, 0x1, 0x7):
+            z = 1 if (result & 0xFF) == 0 else 0
+            n = 1 if (result & 0x80) else 0
+            s["flags"] = [z, n, carry, overflow]
+        s["pc"] = next_pc
+        return s
+
+    def run(self, state, max_cycles=200):
+        s = state
+        cycles = 0
+        while not s["halted"] and cycles < max_cycles:
+            s = self.step(s)
+            cycles += 1
+        return s, cycles
+
+
+def _encode_instr(opcode, rd, rs, imm):
+    return ((opcode & 0xF) << 12) | ((rd & 0x3) << 10) | ((rs & 0x3) << 8) | (imm & 0xFF)
+
+
+def _w16(mem, addr, value):
+    mem[addr] = (value >> 8) & 0xFF
+    mem[addr + 1] = value & 0xFF
+
+
+PROGRAM_MIN_BYTES = 0x84  # code 0x00..0x1F + data 0x80..0x83
+
+
+def builtin_program(addr_bits: int) -> Tuple[List[int], int]:
+    """Sum 5+4+3+2+1 via a loop. Returns (mem, expected_result_at_0x83).
+
+    Compact layout: code at 0x00..0x1F (32 bytes), data at 0x80..0x83 (4 bytes).
+    Total footprint 132 bytes -- fits within scratchpad (256 B) and larger.
+    Requires addr_bits >= 8.
+    """
+    if (1 << addr_bits) < PROGRAM_MIN_BYTES:
+        raise ValueError(f"addr_bits={addr_bits} too small for builtin program")
+    mem = [0] * (1 << addr_bits)
+    mem[0x80] = 5  # initial counter
+    mem[0x81] = 1  # decrement
+    mem[0x82] = 0  # zero (for compare and accumulator init)
+    # mem[0x83] is the output
+    _w16(mem, 0x0000, _encode_instr(0xA, 1, 0, 0)); _w16(mem, 0x0002, 0x0080)
+    _w16(mem, 0x0004, _encode_instr(0xA, 2, 0, 0)); _w16(mem, 0x0006, 0x0081)
+    _w16(mem, 0x0008, _encode_instr(0xA, 3, 0, 0)); _w16(mem, 0x000A, 0x0082)
+    _w16(mem, 0x000C, _encode_instr(0xA, 0, 0, 0)); _w16(mem, 0x000E, 0x0082)
+    _w16(mem, 0x0010, _encode_instr(0x0, 0, 1, 0))
+    _w16(mem, 0x0012, _encode_instr(0x1, 1, 2, 0))
+    _w16(mem, 0x0014, _encode_instr(0x9, 1, 3, 0))
+    _w16(mem, 0x0016, _encode_instr(0xD, 0, 0, 0x01)); _w16(mem, 0x0018, 0x0010)
+    _w16(mem, 0x001A, _encode_instr(0xB, 0, 0, 0)); _w16(mem, 0x001C, 0x0083)
+    _w16(mem, 0x001E, _encode_instr(0xF, 0, 0, 0))
+    return mem, 15
+
+
+# ---------------------------------------------------------------------------
+# Eval driver
+# ---------------------------------------------------------------------------
+
+def _file_fingerprint(path: Path) -> str:
+    """Stable cache key for a safetensors file: sha256 of its content.
+
+    Hashes are content-addressed so renaming a file doesn't blow the cache,
+    but mtime-only would re-key on every clone of the repo. The sha256 of a
+    30 MB safetensors finishes in tens of milliseconds — small compared to
+    a 5,900-test fitness run.
+    """
+    import hashlib
+    h = hashlib.sha256()
+    with open(path, "rb") as f:
+        for chunk in iter(lambda: f.read(1 << 20), b""):
+            h.update(chunk)
+    return h.hexdigest()
+
+
+def _cache_key(path: Path, opts: Dict[str, Any]) -> str:
+    """Cache key combining file content with the relevant evaluation options."""
+    fp = _file_fingerprint(path)
+    opt_str = json.dumps(opts, sort_keys=True)
+    import hashlib
+    suffix = hashlib.sha256(opt_str.encode("utf-8")).hexdigest()[:8]
+    return f"{fp}_{suffix}"
+
+
+def _load_cache(cache_dir: Path, key: str) -> Dict[str, Any] | None:
+    p = cache_dir / f"{key}.json"
+    if not p.exists():
+        return None
+    try:
+        return json.loads(p.read_text(encoding="utf-8"))
+    except (json.JSONDecodeError, OSError):
+        return None
+
+
+def _save_cache(cache_dir: Path, key: str, payload: Dict[str, Any]) -> None:
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    p = cache_dir / f"{key}.json"
+    try:
+        p.write_text(json.dumps(payload, indent=2, default=str), encoding="utf-8")
+    except OSError:
+        pass
+
+
+def list_safetensors(path: Path) -> List[Path]:
+    if path.is_file():
+        return [path]
+    if path.is_dir():
+        return sorted(p for p in path.glob("*.safetensors") if p.is_file())
+    return []
+
+
+def evaluate_one(path: Path, device: str, pop_size: int, debug: bool, run_cpu_program: bool) -> Dict:
+    out: Dict = {"path": str(path), "filename": path.name}
+    # The standalone machines (neural_subleq8, neural_rv32) carry their own
+    # circuit inventory and are verified by machines.py, not the gate-fitness
+    # suite; skip them cleanly rather than error on missing standard gates.
+    with safe_open(str(path), framework="pt") as f:
+        meta = f.metadata() or {}
+    if meta.get("machine"):
+        out.update(status="SKIP", machine=meta["machine"],
+                   note="standalone machine — verify with machines.py")
+        return out
+    try:
+        tensors = load_model(str(path))
+    except Exception as e:
+        out.update(error=f"load failed: {e}", status="ERROR")
+        return out
+
+    manifest = get_manifest(tensors)
+    out.update(
+        size_mb=path.stat().st_size / (1024 * 1024),
+        tensors=len(tensors),
+        params=sum(t.numel() for t in tensors.values()),
+        manifest=manifest,
+    )
+
+    # Move to device
+    tensors = {k: v.to(device) for k, v in tensors.items()}
+
+    try:
+        evaluator = BatchedFitnessEvaluator(device=device, model_path=str(path), tensors=tensors)
+        population = create_population(tensors, pop_size=pop_size, device=device)
+        t0 = time.perf_counter()
+        fitness = evaluator.evaluate(population, debug=debug)
+        elapsed = time.perf_counter() - t0
+        f0 = float(fitness[0].item()) if pop_size == 1 else float(fitness.mean().item())
+        out.update(
+            fitness=f0,
+            total_tests=evaluator.total_tests,
+            elapsed_s=elapsed,
+            categories={k: (float(v[0]), int(v[1])) for k, v in evaluator.category_scores.items()},
+            status="PASS" if f0 >= 0.9999 else "FAIL",
+        )
+    except Exception as e:
+        out.update(error=f"eval failed: {type(e).__name__}: {e}", status="ERROR")
+        return out
+
+    # Optional: CPU program test (8-bit CPU primitives are in every variant)
+    if run_cpu_program:
+        if manifest["memory_bytes"] >= PROGRAM_MIN_BYTES:
+            try:
+                cpu_tensors = {k: v.cpu() for k, v in tensors.items()}
+                cpu = GenericThresholdCPU(cpu_tensors)
+                mem, expected = builtin_program(manifest["addr_bits"])
+                state = {"pc": 0, "regs": [0] * 4, "flags": [0] * 4, "mem": mem, "halted": False}
+                t0 = time.perf_counter()
+                final, cycles = cpu.run(state, max_cycles=200)
+                cpu_elapsed = time.perf_counter() - t0
+                got = final["mem"][0x83]
+                out["cpu_program"] = {
+                    "ok": got == expected,
+                    "got": got,
+                    "expected": expected,
+                    "cycles": cycles,
+                    "elapsed_s": cpu_elapsed,
+                }
+                if got != expected:
+                    out["status"] = "FAIL"
+            except Exception as e:
+                out["cpu_program"] = {"error": str(e)}
+        else:
+            out["cpu_program"] = {"skipped": f"mem={manifest['memory_bytes']}B < {PROGRAM_MIN_BYTES}"}
+
+        # Wider-ALU chain test for 16/32-bit variants
+        bits = manifest["data_bits"]
+        if bits in (16, 32):
+            try:
+                alu_tensors = {k: v.cpu() for k, v in tensors.items()}
+                alu = GenericThresholdALU(alu_tensors, bits)
+                t0 = time.perf_counter()
+                if bits == 16:
+                    x, y = 1234, 5678
+                    z, _ = alu.add_n(x, y, 16);          assert z == (x + y) & 0xFFFF
+                    w, _ = alu.sub_n(z, x, 16);          assert w == (z - x) & 0xFFFF, (w, z - x)
+                    gt = alu.cmp_n(z, x, "greaterthan", 16); assert gt == 1
+                    lt = alu.cmp_n(x, z, "lessthan",   16);  assert lt == 1
+                    eq = alu.cmp_n(w, y, "eq",         16);  assert eq == 1
+                    p   = alu.mul_n(123, 5, 16);          assert p == (123 * 5) & 0xFFFF
+                else:  # 32
+                    x, y = 1_000_000, 999_000
+                    z, _ = alu.sub_n(x, y, 32);          assert z == 1_000
+                    s, _ = alu.add_n(z, x, 32);          assert s == 1_001_000
+                    p    = alu.mul_n(z, 100, 32);        assert p == 100_000
+                    gt = alu.cmp_n(x, y, "greaterthan", 32); assert gt == 1
+                    lt = alu.cmp_n(y, x, "lessthan",   32);  assert lt == 1
+                    eq = alu.cmp_n(p, 100_000, "equality", 32); assert eq == 1
+                chain_dt = time.perf_counter() - t0
+                out[f"alu_chain_{bits}"] = {"ok": True, "elapsed_s": chain_dt}
+            except AssertionError as e:
+                out[f"alu_chain_{bits}"] = {"ok": False, "error": f"chain mismatch: {e}"}
+                out["status"] = "FAIL"
+            except Exception as e:
+                out[f"alu_chain_{bits}"] = {"ok": False, "error": f"{type(e).__name__}: {e}"}
+                out["status"] = "FAIL"
+
+    return out
+
+
+def print_row(r: Dict, show_cpu: bool) -> None:
+    if r.get("status") == "SKIP":
+        print(f"  {r['filename']:<48}  SKIP  ({r.get('machine', 'machine')} — verify with machines.py)")
+        return
+    if "error" in r:
+        print(f"  {r['filename']:<48}  ERROR: {r['error'][:80]}")
+        return
+    m = r["manifest"]
+    fit = f"{r['fitness']:.4f}" if r.get("fitness") is not None else "n/a"
+    cpu_col = ""
+    if show_cpu and "cpu_program" in r:
+        cp = r["cpu_program"]
+        if cp.get("ok"):
+            cpu_col = f"  CPU OK ({cp['cycles']}cyc/{cp['elapsed_s']:.1f}s)"
+        elif "skipped" in cp:
+            cpu_col = f"  CPU SKIP"
+        elif "error" in cp:
+            cpu_col = f"  CPU ERR"
+        else:
+            cpu_col = f"  CPU FAIL ({cp.get('got')}!={cp.get('expected')})"
+    chain_col = ""
+    if show_cpu:
+        for bits in (16, 32):
+            key = f"alu_chain_{bits}"
+            if key in r:
+                ch = r[key]
+                if ch.get("ok"):
+                    chain_col = f"  ALU{bits} OK ({ch['elapsed_s']:.2f}s)"
+                else:
+                    chain_col = f"  ALU{bits} FAIL"
+    print(
+        f"  {r['filename']:<48}  d={m['data_bits']:>2}b a={m['addr_bits']:>2}b "
+        f"mem={m['memory_bytes']:>6}B  size={r['size_mb']:>6.1f}MB  "
+        f"params={r['params']:>10,}  fit={fit:>6}  tests={r['total_tests']:>5}  "
+        f"{r['status']:>5}{cpu_col}{chain_col}"
+    )
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Variant-agnostic eval harness")
+    parser.add_argument("path", help="Path to .safetensors file or directory of files")
+    parser.add_argument("--device", default="cpu", help="cpu (default) or cuda")
+    parser.add_argument("--pop_size", type=int, default=1)
+    parser.add_argument("--debug", action="store_true", help="Per-circuit detail per file")
+    parser.add_argument("--cpu-program", action="store_true",
+                        help="Also run a small assembled program through the threshold CPU "
+                             "(any variant with >= 132 B memory), plus a chained 16- or "
+                             "32-bit ALU sequence on wider variants")
+    parser.add_argument("--json", action="store_true", help="Emit JSON results to stdout instead of a table")
+    parser.add_argument("--cache-dir", default=".eval_cache",
+                        help="Directory for hash-keyed result cache "
+                             "(default: ./.eval_cache). Set to '' to disable.")
+    parser.add_argument("--no-cache", action="store_true",
+                        help="Disable the result cache for this run.")
+    args = parser.parse_args()
+
+    files = list_safetensors(Path(args.path))
+    if not files:
+        print(f"No .safetensors files found under {args.path}", file=sys.stderr)
+        return 2
+
+    print(f"Evaluating {len(files)} file(s) on {args.device}\n")
+    cache_enabled = bool(args.cache_dir) and not args.no_cache
+    cache_dir = Path(args.cache_dir) if cache_enabled else None
+    cache_opts = {
+        "device": args.device,
+        "pop_size": args.pop_size,
+        "cpu_program": bool(args.cpu_program),
+    }
+    cache_hits = 0
+    results = []
+    fail_count = 0
+    for f in files:
+        print(f"=== {f.name}")
+        cached = None
+        key = None
+        if cache_enabled:
+            try:
+                key = _cache_key(f, cache_opts)
+                cached = _load_cache(cache_dir, key)
+            except OSError:
+                cached = None
+        if cached is not None:
+            r = cached
+            cache_hits += 1
+            print(f"   (cache hit)")
+        else:
+            r = evaluate_one(f, device=args.device, pop_size=args.pop_size,
+                             debug=args.debug, run_cpu_program=args.cpu_program)
+            # Don't cache ERROR results: a transient failure (OOM, interrupt)
+            # would otherwise be replayed on every rerun of the same file.
+            if cache_enabled and key is not None and r.get("status") != "ERROR":
+                _save_cache(cache_dir, key, r)
+        results.append(r)
+        print_row(r, show_cpu=args.cpu_program)
+        if r.get("status") not in ("PASS", "SKIP"):
+            fail_count += 1
+
+    if args.json:
+        # Make it serialisable
+        for r in results:
+            r["manifest"] = {k: (int(v) if isinstance(v, float) and v.is_integer() else v)
+                             for k, v in r.get("manifest", {}).items()}
+        print(json.dumps(results, indent=2, default=str))
+        return fail_count
+
+    # Summary
+    print()
+    print("=" * 100)
+    print(" SUMMARY")
+    print("=" * 100)
+    for r in results:
+        print_row(r, show_cpu=args.cpu_program)
+
+    print()
+    if fail_count == 0:
+        print(f"ALL {len(files)} variants PASS")
+    else:
+        print(f"{fail_count}/{len(files)} variants FAIL")
+    if cache_enabled:
+        print(f"(cache: {cache_hits}/{len(files)} hits, dir={cache_dir})")
+    return fail_count
+
+
+if __name__ == "__main__":
+    sys.exit(main())

llm_integration/baseline.py

@@ -0,0 +1,221 @@
+"""
+Baseline evaluation: Vanilla SmolLM2-360M on arithmetic
+"""
+
+import torch
+import random
+import re
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+DEVICE = "cuda"
+MODEL_ID = "HuggingFaceTB/SmolLM2-360M-Instruct"
+
+SYSTEM_PROMPT = """You are a calculator. Output only the numeric answer. No words, no explanation, just digits. Examples:
+User: 5 + 3
+Assistant: 8
+User: 12 * 7
+Assistant: 84
+User: 100 > 50
+Assistant: 1
+User: 25 < 10
+Assistant: 0"""
+
+
+def load_model():
+    print(f"Loading {MODEL_ID}...")
+    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+    tokenizer.padding_side = "left"
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    model = AutoModelForCausalLM.from_pretrained(
+        MODEL_ID,
+        torch_dtype=torch.float16,
+        device_map=DEVICE
+    )
+    model.eval()
+    print(f"  Loaded. Parameters: {sum(p.numel() for p in model.parameters()):,}")
+    return model, tokenizer
+
+
+def format_prompt(tokenizer, op_str):
+    messages = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": op_str}
+    ]
+    return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+
+def generate_batch(model, tokenizer, prompts, max_new_tokens=16):
+    inputs = tokenizer(prompts, return_tensors="pt", padding=True).to(DEVICE)
+    with torch.no_grad():
+        outputs = model.generate(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=False,
+            pad_token_id=tokenizer.eos_token_id
+        )
+    responses = []
+    for i, output in enumerate(outputs):
+        response = tokenizer.decode(output[inputs.input_ids.shape[1]:], skip_special_tokens=True)
+        responses.append(response.strip())
+    return responses
+
+
+def extract_answer(text):
+    """Generous extraction - find any number in output"""
+    text = text.strip().lower()
+    if not text:
+        return None
+
+    # Handle Yes/No for comparisons
+    if text in ['yes', 'true', '1']:
+        return 1
+    if text in ['no', 'false', '0']:
+        return 0
+    if text.startswith('yes'):
+        return 1
+    if text.startswith('no'):
+        return 0
+
+    # Find all numbers, take the LAST one (most likely the answer)
+    numbers = re.findall(r'-?\d+', text)
+    if numbers:
+        return int(numbers[-1])
+    return None
+
+
+def ground_truth(a, b, op):
+    """Compute expected result (8-bit where applicable)"""
+    if op == 'add':
+        return (a + b) & 0xFF
+    elif op == 'sub':
+        return (a - b) & 0xFF
+    elif op == 'mul':
+        return (a * b) & 0xFF
+    elif op == 'div':
+        return a // b if b != 0 else 0
+    elif op == 'and':
+        return a & b
+    elif op == 'or':
+        return a | b
+    elif op == 'xor':
+        return a ^ b
+    elif op == 'gt':
+        return 1 if a > b else 0
+    elif op == 'lt':
+        return 1 if a < b else 0
+    elif op == 'eq':
+        return 1 if a == b else 0
+    elif op == 'ge':
+        return 1 if a >= b else 0
+    elif op == 'le':
+        return 1 if a <= b else 0
+    else:
+        raise ValueError(f"Unknown op: {op}")
+
+
+def op_to_str(a, b, op):
+    """Convert operation to natural string"""
+    symbols = {
+        'add': '+', 'sub': '-', 'mul': '*', 'div': '/',
+        'and': '&', 'or': '|', 'xor': '^',
+        'gt': '>', 'lt': '<', 'eq': '==', 'ge': '>=', 'le': '<='
+    }
+    return f"{a} {symbols[op]} {b}"
+
+
+def evaluate(model, tokenizer, n_samples=1000, batch_size=32, ops=None):
+    if ops is None:
+        ops = ['add', 'sub', 'mul', 'gt', 'lt', 'eq']
+
+    results = {op: {'correct': 0, 'total': 0} for op in ops}
+    all_correct = 0
+    all_total = 0
+
+    samples = []
+    for _ in range(n_samples):
+        a = random.randint(0, 255)
+        b = random.randint(0, 255)
+        if 'div' in ops and random.random() < 0.1:
+            op = 'div'
+            b = random.randint(1, 255)  # avoid div by zero
+        else:
+            op = random.choice([o for o in ops if o != 'div'])
+        samples.append((a, b, op))
+
+    print(f"\nEvaluating {n_samples} samples (batch_size={batch_size})...")
+
+    for batch_start in range(0, n_samples, batch_size):
+        batch = samples[batch_start:batch_start + batch_size]
+        prompts = [format_prompt(tokenizer, op_to_str(a, b, op)) for a, b, op in batch]
+        responses = generate_batch(model, tokenizer, prompts)
+
+        for (a, b, op), response in zip(batch, responses):
+            expected = ground_truth(a, b, op)
+            extracted = extract_answer(response)
+
+            correct = (extracted == expected)
+            results[op]['total'] += 1
+            all_total += 1
+            if correct:
+                results[op]['correct'] += 1
+                all_correct += 1
+
+        if (batch_start + batch_size) % 200 == 0 or batch_start + batch_size >= n_samples:
+            pct = 100 * all_correct / all_total
+            print(f"  Progress: {min(batch_start + batch_size, n_samples)}/{n_samples} | Accuracy: {pct:.2f}%")
+
+    return results, all_correct, all_total
+
+
+def main():
+    random.seed(42)
+    torch.manual_seed(42)
+
+    model, tokenizer = load_model()
+
+    # Quick sanity check
+    print("\nSanity check (5 examples):")
+    test_cases = [
+        ("5 + 3", 8),
+        ("100 - 37", 63),
+        ("12 * 11", 132),
+        ("50 > 30", 1),
+        ("25 < 10", 0),
+    ]
+    prompts = [format_prompt(tokenizer, q) for q, _ in test_cases]
+    responses = generate_batch(model, tokenizer, prompts)
+    for (q, expected), response in zip(test_cases, responses):
+        extracted = extract_answer(response)
+        status = "OK" if extracted == expected else "FAIL"
+        print(f"  {q} = {expected} | Model: '{response}' -> {extracted} [{status}]")
+
+    # Full evaluation
+    print("\n" + "=" * 60)
+    print(" BASELINE EVALUATION")
+    print("=" * 60)
+
+    ops = ['add', 'sub', 'mul', 'gt', 'lt', 'eq']
+    results, correct, total = evaluate(model, tokenizer, n_samples=2000, batch_size=64, ops=ops)
+
+    print("\n" + "=" * 60)
+    print(" RESULTS BY OPERATION")
+    print("=" * 60)
+    for op in ops:
+        r = results[op]
+        pct = 100 * r['correct'] / r['total'] if r['total'] > 0 else 0
+        print(f"  {op:6}: {r['correct']:4}/{r['total']:4} ({pct:6.2f}%)")
+
+    print("\n" + "=" * 60)
+    print(" OVERALL")
+    print("=" * 60)
+    fitness = correct / total
+    print(f"  Correct: {correct}/{total}")
+    print(f"  Fitness: {fitness:.4f} ({100*fitness:.2f}%)")
+    print("=" * 60)
+
+    return fitness
+
+
+if __name__ == "__main__":
+    main()

llm_integration/circuits.py

@@ -0,0 +1,320 @@
+"""
+Frozen threshold circuit wrapper for LLM integration.
+Loads safetensors and provides differentiable-compatible execution.
+"""
+
+import torch
+import torch.nn as nn
+from safetensors import safe_open
+from typing import Dict, Tuple
+
+MODEL_PATH = "D:/8bit-threshold-computer/neural_computer.safetensors"
+
+
+def heaviside(x: torch.Tensor) -> torch.Tensor:
+    """Standard Heaviside step function."""
+    return (x >= 0).float()
+
+
+class HeavisideSTE(torch.autograd.Function):
+    """Heaviside with straight-through estimator for gradients."""
+    @staticmethod
+    def forward(ctx, x):
+        return (x >= 0).float()
+
+    @staticmethod
+    def backward(ctx, grad_output):
+        return grad_output
+
+
+def heaviside_ste(x: torch.Tensor) -> torch.Tensor:
+    """Heaviside with STE gradient."""
+    return HeavisideSTE.apply(x)
+
+
+class FrozenThresholdCircuits(nn.Module):
+    """
+    Wrapper for frozen threshold logic circuits.
+    All weights are frozen - no gradients flow through circuit internals.
+    Gradients flow through inputs/outputs via STE.
+    """
+
+    def __init__(self, model_path: str = MODEL_PATH, device: str = 'cuda'):
+        super().__init__()
+        self.device = device
+        self.weights = {}
+        self._load_weights(model_path)
+
+    def _load_weights(self, path: str):
+        """Load weights from safetensors file."""
+        with safe_open(path, framework='pt') as f:
+            for name in f.keys():
+                tensor = f.get_tensor(name).to(self.device).float()
+                self.weights[name] = tensor
+
+    def _gate(self, inputs: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor) -> torch.Tensor:
+        """Execute single threshold gate with STE."""
+        weight = weight.view(-1)
+        bias = bias.view(-1)
+        pre_activation = (inputs * weight).sum(dim=-1) + bias
+        return heaviside_ste(pre_activation)
+
+    def _xor(self, a: torch.Tensor, b: torch.Tensor, prefix: str) -> torch.Tensor:
+        """XOR via OR-NAND-AND pattern (2 layers)."""
+        inputs = torch.stack([a, b], dim=-1)
+
+        w_or = self.weights[f'{prefix}.layer1.or.weight']
+        b_or = self.weights[f'{prefix}.layer1.or.bias']
+        w_nand = self.weights[f'{prefix}.layer1.nand.weight']
+        b_nand = self.weights[f'{prefix}.layer1.nand.bias']
+
+        h_or = self._gate(inputs, w_or, b_or)
+        h_nand = self._gate(inputs, w_nand, b_nand)
+
+        hidden = torch.stack([h_or, h_nand], dim=-1)
+        w2 = self.weights[f'{prefix}.layer2.weight']
+        b2 = self.weights[f'{prefix}.layer2.bias']
+
+        return self._gate(hidden, w2, b2)
+
+    def _full_adder(self, a: torch.Tensor, b: torch.Tensor, cin: torch.Tensor,
+                    prefix: str) -> Tuple[torch.Tensor, torch.Tensor]:
+        """Full adder: sum and carry out."""
+        ha1_sum = self._xor(a, b, f'{prefix}.ha1.sum')
+
+        inp_carry1 = torch.stack([a, b], dim=-1)
+        w_c1 = self.weights[f'{prefix}.ha1.carry.weight']
+        b_c1 = self.weights[f'{prefix}.ha1.carry.bias']
+        ha1_carry = self._gate(inp_carry1, w_c1, b_c1)
+
+        ha2_sum = self._xor(ha1_sum, cin, f'{prefix}.ha2.sum')
+
+        inp_carry2 = torch.stack([ha1_sum, cin], dim=-1)
+        w_c2 = self.weights[f'{prefix}.ha2.carry.weight']
+        b_c2 = self.weights[f'{prefix}.ha2.carry.bias']
+        ha2_carry = self._gate(inp_carry2, w_c2, b_c2)
+
+        inp_cout = torch.stack([ha1_carry, ha2_carry], dim=-1)
+        w_cout = self.weights[f'{prefix}.carry_or.weight']
+        b_cout = self.weights[f'{prefix}.carry_or.bias']
+        cout = self._gate(inp_cout, w_cout, b_cout)
+
+        return ha2_sum, cout
+
+    def add_8bit(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
+        """
+        8-bit ripple carry addition.
+
+        Args:
+            a_bits: [batch, 8] MSB-first
+            b_bits: [batch, 8] MSB-first
+
+        Returns:
+            result_bits: [batch, 8] MSB-first
+            carry_out: [batch] final carry
+        """
+        batch_size = a_bits.shape[0]
+        carry = torch.zeros(batch_size, device=self.device)
+        result_bits = []
+
+        for bit in range(8):
+            bit_idx = 7 - bit
+            s, carry = self._full_adder(
+                a_bits[:, bit_idx],
+                b_bits[:, bit_idx],
+                carry,
+                f'arithmetic.ripplecarry8bit.fa{bit}'
+            )
+            result_bits.insert(0, s)
+
+        result = torch.stack(result_bits, dim=1)
+        return result, carry
+
+    def sub_8bit(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
+        """
+        8-bit subtraction via two's complement: A - B = A + (~B) + 1
+
+        Args:
+            a_bits: [batch, 8] MSB-first
+            b_bits: [batch, 8] MSB-first
+
+        Returns:
+            result_bits: [batch, 8] MSB-first
+            borrow_out: [batch] (inverted carry)
+        """
+        b_inv = 1.0 - b_bits
+        batch_size = a_bits.shape[0]
+        carry = torch.ones(batch_size, device=self.device)
+        result_bits = []
+
+        for bit in range(8):
+            bit_idx = 7 - bit
+            s, carry = self._full_adder(
+                a_bits[:, bit_idx],
+                b_inv[:, bit_idx],
+                carry,
+                f'arithmetic.ripplecarry8bit.fa{bit}'
+            )
+            result_bits.insert(0, s)
+
+        result = torch.stack(result_bits, dim=1)
+        borrow = 1.0 - carry
+        return result, borrow
+
+    def mul_8bit(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> torch.Tensor:
+        """
+        8-bit multiplication via shift-add (software implementation using adder circuits).
+        Only keeps low 8 bits of result (matches 8-bit wrap behavior).
+
+        Args:
+            a_bits: [batch, 8] MSB-first
+            b_bits: [batch, 8] MSB-first
+
+        Returns:
+            result_bits: [batch, 8] MSB-first (low 8 bits of product)
+        """
+        batch_size = a_bits.shape[0]
+
+        acc = torch.zeros(batch_size, 8, device=self.device)
+
+        for i in range(8):
+            b_bit = b_bits[:, 7 - i]
+            pp = a_bits * b_bit.unsqueeze(1)
+
+            shifted_pp = torch.zeros(batch_size, 8, device=self.device)
+            for j in range(8):
+                dst_idx = j + i
+                if dst_idx < 8:
+                    shifted_pp[:, 7 - dst_idx] = pp[:, 7 - j]
+
+            acc, _ = self.add_8bit(acc, shifted_pp)
+
+        return acc
+
+    def compare_gt(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> torch.Tensor:
+        """A > B comparison."""
+        inputs = torch.cat([a_bits, b_bits], dim=-1)
+        w = self.weights['arithmetic.greaterthan8bit.weight'].view(-1)
+        b = self.weights['arithmetic.greaterthan8bit.bias'].view(-1)
+        return heaviside_ste((inputs * w).sum(dim=-1) + b)
+
+    def compare_lt(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> torch.Tensor:
+        """A < B comparison."""
+        inputs = torch.cat([a_bits, b_bits], dim=-1)
+        w = self.weights['arithmetic.lessthan8bit.weight'].view(-1)
+        b = self.weights['arithmetic.lessthan8bit.bias'].view(-1)
+        return heaviside_ste((inputs * w).sum(dim=-1) + b)
+
+    def compare_eq(self, a_bits: torch.Tensor, b_bits: torch.Tensor) -> torch.Tensor:
+        """A == B comparison (two-layer)."""
+        inputs = torch.cat([a_bits, b_bits], dim=-1)
+        prefix = 'arithmetic.equality8bit'
+
+        w_geq = self.weights[f'{prefix}.layer1.geq.weight'].view(-1)
+        b_geq = self.weights[f'{prefix}.layer1.geq.bias'].view(-1)
+        w_leq = self.weights[f'{prefix}.layer1.leq.weight'].view(-1)
+        b_leq = self.weights[f'{prefix}.layer1.leq.bias'].view(-1)
+
+        h_geq = heaviside_ste((inputs * w_geq).sum(dim=-1) + b_geq)
+        h_leq = heaviside_ste((inputs * w_leq).sum(dim=-1) + b_leq)
+
+        hidden = torch.stack([h_geq, h_leq], dim=-1)
+        w2 = self.weights[f'{prefix}.layer2.weight'].view(-1)
+        b2 = self.weights[f'{prefix}.layer2.bias'].view(-1)
+
+        return heaviside_ste((hidden * w2).sum(dim=-1) + b2)
+
+    def forward(self, a_bits: torch.Tensor, b_bits: torch.Tensor,
+                op_onehot: torch.Tensor) -> torch.Tensor:
+        """
+        Execute operation based on one-hot selector.
+        Uses soft routing during training for gradient flow.
+
+        Args:
+            a_bits: [batch, 8] operand A
+            b_bits: [batch, 8] operand B
+            op_onehot: [batch, 6] one-hot operation selector
+                       [add, sub, mul, gt, lt, eq]
+
+        Returns:
+            result_bits: [batch, 8] result (comparisons in bit 7, rest zeros)
+        """
+        batch_size = a_bits.shape[0]
+
+        add_result, _ = self.add_8bit(a_bits, b_bits)
+        sub_result, _ = self.sub_8bit(a_bits, b_bits)
+        mul_result = self.mul_8bit(a_bits, b_bits)
+
+        gt_result = self.compare_gt(a_bits, b_bits)
+        lt_result = self.compare_lt(a_bits, b_bits)
+        eq_result = self.compare_eq(a_bits, b_bits)
+
+        cmp_expanded = torch.zeros(batch_size, 8, device=self.device)
+
+        gt_expanded = cmp_expanded.clone()
+        gt_expanded[:, 7] = gt_result
+
+        lt_expanded = cmp_expanded.clone()
+        lt_expanded[:, 7] = lt_result
+
+        eq_expanded = cmp_expanded.clone()
+        eq_expanded[:, 7] = eq_result
+
+        results = torch.stack([
+            add_result,
+            sub_result,
+            mul_result,
+            gt_expanded,
+            lt_expanded,
+            eq_expanded
+        ], dim=1)
+
+        op_weights = op_onehot.unsqueeze(-1)
+        output = (results * op_weights).sum(dim=1)
+
+        return output
+
+
+if __name__ == "__main__":
+    print("Testing frozen circuits...")
+
+    circuits = FrozenThresholdCircuits(device='cuda')
+    print(f"Loaded {len(circuits.weights)} tensors")
+
+    a = torch.tensor([[0, 0, 0, 0, 0, 1, 0, 1]], device='cuda', dtype=torch.float32)
+    b = torch.tensor([[0, 0, 0, 0, 0, 0, 1, 1]], device='cuda', dtype=torch.float32)
+
+    result, carry = circuits.add_8bit(a, b)
+    val = sum(int(result[0, i].item()) << (7 - i) for i in range(8))
+    print(f"5 + 3 = {val} (expected 8)")
+
+    a = torch.tensor([[0, 1, 1, 0, 0, 1, 0, 0]], device='cuda', dtype=torch.float32)
+    b = torch.tensor([[0, 0, 1, 0, 0, 1, 0, 1]], device='cuda', dtype=torch.float32)
+    result, _ = circuits.sub_8bit(a, b)
+    val = sum(int(result[0, i].item()) << (7 - i) for i in range(8))
+    print(f"100 - 37 = {val} (expected 63)")
+
+    a = torch.tensor([[0, 0, 0, 0, 1, 1, 0, 0]], device='cuda', dtype=torch.float32)
+    b = torch.tensor([[0, 0, 0, 0, 1, 0, 1, 1]], device='cuda', dtype=torch.float32)
+    result = circuits.mul_8bit(a, b)
+    val = sum(int(result[0, i].item()) << (7 - i) for i in range(8))
+    print(f"12 * 11 = {val} (expected 132)")
+
+    a = torch.tensor([[0, 0, 1, 1, 0, 0, 1, 0]], device='cuda', dtype=torch.float32)
+    b = torch.tensor([[0, 0, 0, 1, 1, 1, 1, 0]], device='cuda', dtype=torch.float32)
+    gt = circuits.compare_gt(a, b)
+    lt = circuits.compare_lt(a, b)
+    eq = circuits.compare_eq(a, b)
+    print(f"50 > 30: {int(gt[0].item())} (expected 1)")
+    print(f"50 < 30: {int(lt[0].item())} (expected 0)")
+    print(f"50 == 30: {int(eq[0].item())} (expected 0)")
+
+    print("\nTesting batched forward...")
+    batch_a = torch.randint(0, 2, (16, 8), device='cuda', dtype=torch.float32)
+    batch_b = torch.randint(0, 2, (16, 8), device='cuda', dtype=torch.float32)
+    op = torch.zeros(16, 6, device='cuda')
+    op[:, 0] = 1.0
+
+    result = circuits(batch_a, batch_b, op)
+    print(f"Batch output shape: {result.shape}")
+    print("Done.")

llm_integration/fitness.py

@@ -0,0 +1,218 @@
+"""
+Shared fitness function for threshold circuit LLM integration.
+Randomized tests, no answer supervision - fitness IS the training signal.
+"""
+
+import torch
+import random
+from typing import Callable, Dict, Tuple, List
+
+OPERATIONS = ['add', 'sub', 'mul', 'gt', 'lt', 'eq']
+
+def ground_truth(a: int, b: int, op: str) -> int:
+    """Compute expected result (8-bit arithmetic)."""
+    if op == 'add':
+        return (a + b) & 0xFF
+    elif op == 'sub':
+        return (a - b) & 0xFF
+    elif op == 'mul':
+        return (a * b) & 0xFF
+    elif op == 'gt':
+        return 1 if a > b else 0
+    elif op == 'lt':
+        return 1 if a < b else 0
+    elif op == 'eq':
+        return 1 if a == b else 0
+    else:
+        raise ValueError(f"Unknown op: {op}")
+
+
+def int_to_bits(val: int, n_bits: int = 8) -> torch.Tensor:
+    """Convert integer to bit tensor (MSB first)."""
+    bits = torch.zeros(n_bits)
+    for i in range(n_bits):
+        bits[n_bits - 1 - i] = (val >> i) & 1
+    return bits
+
+
+def bits_to_int(bits: torch.Tensor) -> int:
+    """Convert bit tensor to integer (MSB first)."""
+    val = 0
+    n_bits = bits.shape[-1]
+    for i in range(n_bits):
+        val += int(bits[..., i].item()) << (n_bits - 1 - i)
+    return val
+
+
+def op_to_idx(op: str) -> int:
+    """Convert operation string to index."""
+    return OPERATIONS.index(op)
+
+
+def idx_to_op(idx: int) -> str:
+    """Convert index to operation string."""
+    return OPERATIONS[idx]
+
+
+def generate_batch(batch_size: int, device: str = 'cuda') -> Dict[str, torch.Tensor]:
+    """
+    Generate a batch of random arithmetic problems.
+
+    Returns:
+        Dict with:
+            'a': [batch_size] int tensor of first operands
+            'b': [batch_size] int tensor of second operands
+            'op': [batch_size] int tensor of operation indices
+            'a_bits': [batch_size, 8] bit tensor
+            'b_bits': [batch_size, 8] bit tensor
+            'op_onehot': [batch_size, 6] one-hot operation tensor
+            'expected': [batch_size] int tensor of expected results
+            'expected_bits': [batch_size, 8] bit tensor of expected results
+    """
+    a_vals = torch.randint(0, 256, (batch_size,), device=device)
+    b_vals = torch.randint(0, 256, (batch_size,), device=device)
+    op_indices = torch.randint(0, len(OPERATIONS), (batch_size,), device=device)
+
+    a_bits = torch.zeros(batch_size, 8, device=device)
+    b_bits = torch.zeros(batch_size, 8, device=device)
+    for i in range(8):
+        a_bits[:, 7-i] = (a_vals >> i) & 1
+        b_bits[:, 7-i] = (b_vals >> i) & 1
+
+    op_onehot = torch.zeros(batch_size, len(OPERATIONS), device=device)
+    op_onehot.scatter_(1, op_indices.unsqueeze(1), 1.0)
+
+    expected = torch.zeros(batch_size, dtype=torch.long, device=device)
+    for i in range(batch_size):
+        a, b, op_idx = a_vals[i].item(), b_vals[i].item(), op_indices[i].item()
+        expected[i] = ground_truth(a, b, idx_to_op(op_idx))
+
+    expected_bits = torch.zeros(batch_size, 8, device=device)
+    for i in range(8):
+        expected_bits[:, 7-i] = (expected >> i) & 1
+
+    return {
+        'a': a_vals,
+        'b': b_vals,
+        'op': op_indices,
+        'a_bits': a_bits.float(),
+        'b_bits': b_bits.float(),
+        'op_onehot': op_onehot.float(),
+        'expected': expected,
+        'expected_bits': expected_bits.float(),
+    }
+
+
+def compute_fitness(
+    model_fn: Callable[[torch.Tensor, torch.Tensor, torch.Tensor], torch.Tensor],
+    n_samples: int = 10000,
+    batch_size: int = 256,
+    device: str = 'cuda',
+    return_details: bool = False
+) -> float | Tuple[float, Dict]:
+    """
+    Compute fitness score for a model.
+
+    Args:
+        model_fn: Function that takes (a_bits, b_bits, op_onehot) and returns result_bits
+        n_samples: Number of test cases
+        batch_size: Batch size for evaluation
+        device: Device to run on
+        return_details: If True, return per-operation breakdown
+
+    Returns:
+        Fitness score in [0, 1], optionally with details dict
+    """
+    correct = 0
+    total = 0
+    op_correct = {op: 0 for op in OPERATIONS}
+    op_total = {op: 0 for op in OPERATIONS}
+
+    for _ in range(0, n_samples, batch_size):
+        actual_batch = min(batch_size, n_samples - total)
+        batch = generate_batch(actual_batch, device)
+
+        with torch.no_grad():
+            pred_bits = model_fn(batch['a_bits'], batch['b_bits'], batch['op_onehot'])
+
+        pred_bits_binary = (pred_bits > 0.5).float()
+
+        for i in range(actual_batch):
+            pred_val = 0
+            for j in range(8):
+                pred_val += int(pred_bits_binary[i, j].item()) << (7 - j)
+
+            expected_val = batch['expected'][i].item()
+            op_name = idx_to_op(batch['op'][i].item())
+
+            op_total[op_name] += 1
+            total += 1
+
+            if pred_val == expected_val:
+                correct += 1
+                op_correct[op_name] += 1
+
+    fitness = correct / total if total > 0 else 0.0
+
+    if return_details:
+        details = {
+            'correct': correct,
+            'total': total,
+            'by_op': {
+                op: {
+                    'correct': op_correct[op],
+                    'total': op_total[op],
+                    'accuracy': op_correct[op] / op_total[op] if op_total[op] > 0 else 0.0
+                }
+                for op in OPERATIONS
+            }
+        }
+        return fitness, details
+
+    return fitness
+
+
+def compute_bit_accuracy(pred_bits: torch.Tensor, expected_bits: torch.Tensor) -> float:
+    """Compute per-bit accuracy (for gradient signal analysis)."""
+    pred_binary = (pred_bits > 0.5).float()
+    return (pred_binary == expected_bits).float().mean().item()
+
+
+def compute_loss(pred_bits: torch.Tensor, expected_bits: torch.Tensor) -> torch.Tensor:
+    """Binary cross-entropy loss on output bits."""
+    pred_clamped = pred_bits.clamp(1e-7, 1 - 1e-7)
+    return -((expected_bits * torch.log(pred_clamped) +
+              (1 - expected_bits) * torch.log(1 - pred_clamped))).mean()
+
+
+if __name__ == "__main__":
+    print("Testing fitness module...")
+
+    batch = generate_batch(8, 'cpu')
+    print(f"\nSample batch:")
+    for i in range(4):
+        a, b = batch['a'][i].item(), batch['b'][i].item()
+        op = idx_to_op(batch['op'][i].item())
+        expected = batch['expected'][i].item()
+        print(f"  {a} {op} {b} = {expected}")
+
+    def random_model(a_bits, b_bits, op_onehot):
+        return torch.rand(a_bits.shape[0], 8, device=a_bits.device)
+
+    fitness = compute_fitness(random_model, n_samples=1000, batch_size=100, device='cpu')
+    print(f"\nRandom model fitness: {fitness:.4f} (expected ~0.004 for 8-bit)")
+
+    def perfect_model(a_bits, b_bits, op_onehot):
+        batch_size = a_bits.shape[0]
+        results = torch.zeros(batch_size, 8, device=a_bits.device)
+        for i in range(batch_size):
+            a = sum(int(a_bits[i, j].item()) << (7-j) for j in range(8))
+            b = sum(int(b_bits[i, j].item()) << (7-j) for j in range(8))
+            op_idx = op_onehot[i].argmax().item()
+            result = ground_truth(a, b, idx_to_op(op_idx))
+            for j in range(8):
+                results[i, 7-j] = (result >> j) & 1
+        return results
+
+    fitness = compute_fitness(perfect_model, n_samples=1000, batch_size=100, device='cpu')
+    print(f"Perfect model fitness: {fitness:.4f} (expected 1.0)")

llm_integration/model.py

@@ -0,0 +1,1263 @@
+"""
+Trainable interface layers for frozen threshold circuits.
+BitEncoder, OpRouter, BitDecoder wrap the frozen circuits.
+HiddenStateExtractor and AugmentedArithmeticModel for LLM integration.
+"""
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from circuits import FrozenThresholdCircuits, heaviside_ste
+
+MODEL_ID = 'HuggingFaceTB/SmolLM2-360M-Instruct'
+OPERATIONS = ['add', 'sub', 'mul', 'gt', 'lt', 'eq']
+OP_SYMBOLS = {'add': '+', 'sub': '-', 'mul': '*', 'gt': '>', 'lt': '<', 'eq': '=='}
+
+
+class BitEncoder(nn.Module):
+    """
+    Encodes two 8-bit operands from input representation.
+    Uses residual connection to preserve ground truth bits while allowing learned refinement.
+    """
+
+    def __init__(self, input_dim: int = 16 + 6, hidden_dim: int = 32):
+        super().__init__()
+        self.refine = nn.Sequential(
+            nn.Linear(input_dim, hidden_dim),
+            nn.Tanh(),
+            nn.Linear(hidden_dim, 16),
+        )
+        self.scale = nn.Parameter(torch.tensor(0.0))
+
+    def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+        """
+        Args:
+            x: [batch, input_dim] input with first 16 dims being a_bits, b_bits
+
+        Returns:
+            a_bits: [batch, 8] first operand bits
+            b_bits: [batch, 8] second operand bits
+        """
+        base_bits = x[:, :16]
+        refinement = self.refine(x) * torch.sigmoid(self.scale)
+        bits = base_bits + refinement
+        bits = torch.clamp(bits, 0, 1)
+        hard_bits = heaviside_ste(bits - 0.5)
+        out = hard_bits - bits.detach() + bits
+
+        return out[:, :8], out[:, 8:]
+
+
+class OpRouter(nn.Module):
+    """
+    Routes computation to the appropriate circuit based on input.
+    Outputs soft weights over operations for gradient flow.
+    """
+
+    def __init__(self, input_dim: int = 16 + 6, hidden_dim: int = 32, n_ops: int = 6):
+        super().__init__()
+        self.net = nn.Sequential(
+            nn.Linear(input_dim, hidden_dim),
+            nn.ReLU(),
+            nn.Linear(hidden_dim, n_ops),
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        Args:
+            x: [batch, input_dim] input features
+
+        Returns:
+            op_weights: [batch, n_ops] soft operation weights (softmax)
+        """
+        logits = self.net(x)
+        return F.softmax(logits, dim=-1)
+
+
+class BitDecoder(nn.Module):
+    """
+    Decodes circuit output bits to target representation.
+    For standalone training: outputs soft bits for loss computation.
+    For LLM integration: would project to hidden state delta.
+    """
+
+    def __init__(self, output_dim: int = 8):
+        super().__init__()
+        self.output_dim = output_dim
+
+    def forward(self, result_bits: torch.Tensor) -> torch.Tensor:
+        return result_bits
+
+
+class ThresholdALU(nn.Module):
+    """
+    Complete trainable interface + frozen circuits.
+    Learns to encode inputs, route to circuits, decode outputs.
+    """
+
+    def __init__(self, device: str = 'cuda'):
+        super().__init__()
+        self.device = device
+
+        self.circuits = FrozenThresholdCircuits(device=device)
+
+        for key in self.circuits.weights:
+            self.circuits.weights[key].requires_grad = False
+
+        self.encoder = BitEncoder(input_dim=16 + 6, hidden_dim=64).to(device)
+        self.router = OpRouter(input_dim=16 + 6, hidden_dim=32, n_ops=6).to(device)
+        self.decoder = BitDecoder(output_dim=8).to(device)
+
+    def forward(self, a_bits_in: torch.Tensor, b_bits_in: torch.Tensor,
+                op_onehot: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass through trainable interface + frozen circuits.
+
+        Args:
+            a_bits_in: [batch, 8] input A bits (ground truth for training)
+            b_bits_in: [batch, 8] input B bits (ground truth for training)
+            op_onehot: [batch, 6] one-hot operation selector
+
+        Returns:
+            result_bits: [batch, 8] output bits
+        """
+        x = torch.cat([a_bits_in, b_bits_in, op_onehot], dim=-1)
+
+        a_bits, b_bits = self.encoder(x)
+
+        op_weights = self.router(x)
+
+        result = self.circuits(a_bits, b_bits, op_weights)
+
+        output = self.decoder(result)
+
+        return output
+
+    def forward_direct(self, a_bits: torch.Tensor, b_bits: torch.Tensor,
+                       op_onehot: torch.Tensor) -> torch.Tensor:
+        """
+        Direct forward through circuits (bypass encoder/router for testing).
+        """
+        return self.circuits(a_bits, b_bits, op_onehot)
+
+
+class DirectCircuitModel(nn.Module):
+    """
+    Minimal model that directly uses circuits without learned encoding.
+    For validating that circuits themselves achieve 100% fitness.
+    """
+
+    def __init__(self, device: str = 'cuda'):
+        super().__init__()
+        self.device = device
+        self.circuits = FrozenThresholdCircuits(device=device)
+
+    def forward(self, a_bits: torch.Tensor, b_bits: torch.Tensor,
+                op_onehot: torch.Tensor) -> torch.Tensor:
+        return self.circuits(a_bits, b_bits, op_onehot)
+
+
+class HiddenStateExtractor(nn.Module):
+    """
+    Extracts operands and operation from LLM hidden states.
+    This is the hard part - must learn to parse numbers from embeddings.
+    """
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 256):
+        super().__init__()
+
+        self.a_extractor = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, 8),
+        )
+
+        self.b_extractor = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, 8),
+        )
+
+        self.op_router = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, len(OPERATIONS)),
+        )
+
+    def forward(self, hidden_states: torch.Tensor):
+        """
+        Args:
+            hidden_states: [batch, hidden_dim] from LLM
+
+        Returns:
+            a_bits: [batch, 8]
+            b_bits: [batch, 8]
+            op_logits: [batch, 6]
+        """
+        a_logits = self.a_extractor(hidden_states)
+        b_logits = self.b_extractor(hidden_states)
+        op_logits = self.op_router(hidden_states)
+
+        a_soft = torch.sigmoid(a_logits)
+        b_soft = torch.sigmoid(b_logits)
+
+        a_hard = heaviside_ste(a_logits)
+        b_hard = heaviside_ste(b_logits)
+
+        a_bits = a_hard - a_soft.detach() + a_soft
+        b_bits = b_hard - b_soft.detach() + b_soft
+
+        return a_bits, b_bits, op_logits
+
+
+class AttentionPooling(nn.Module):
+    """
+    Learnable attention pooling over sequence positions.
+    Replaces mean pooling - learns which tokens matter for extraction.
+    """
+
+    def __init__(self, hidden_dim: int = 960, num_heads: int = 4):
+        super().__init__()
+        self.num_heads = num_heads
+        self.head_dim = hidden_dim // num_heads
+
+        self.query = nn.Linear(hidden_dim, hidden_dim)
+        self.key = nn.Linear(hidden_dim, hidden_dim)
+        self.value = nn.Linear(hidden_dim, hidden_dim)
+        self.out_proj = nn.Linear(hidden_dim, hidden_dim)
+
+        self.cls_token = nn.Parameter(torch.randn(1, 1, hidden_dim) * 0.02)
+
+    def forward(self, embeddings: torch.Tensor, mask: torch.Tensor) -> torch.Tensor:
+        """
+        Args:
+            embeddings: [batch, seq_len, hidden_dim]
+            mask: [batch, seq_len] attention mask (1 = attend, 0 = ignore)
+
+        Returns:
+            pooled: [batch, hidden_dim]
+        """
+        batch_size, seq_len, hidden_dim = embeddings.shape
+
+        cls_expanded = self.cls_token.expand(batch_size, -1, -1)
+        embeddings = torch.cat([cls_expanded, embeddings], dim=1)
+
+        cls_mask = torch.ones(batch_size, 1, device=mask.device)
+        mask = torch.cat([cls_mask, mask], dim=1)
+
+        Q = self.query(embeddings[:, :1, :])
+        K = self.key(embeddings)
+        V = self.value(embeddings)
+
+        Q = Q.view(batch_size, 1, self.num_heads, self.head_dim).transpose(1, 2)
+        K = K.view(batch_size, seq_len + 1, self.num_heads, self.head_dim).transpose(1, 2)
+        V = V.view(batch_size, seq_len + 1, self.num_heads, self.head_dim).transpose(1, 2)
+
+        scores = torch.matmul(Q, K.transpose(-2, -1)) / (self.head_dim ** 0.5)
+
+        mask_expanded = mask.unsqueeze(1).unsqueeze(2)
+        scores = scores.masked_fill(mask_expanded == 0, -1e9)
+
+        attn_weights = torch.softmax(scores, dim=-1)
+        attn_weights = torch.nan_to_num(attn_weights, nan=0.0)
+
+        context = torch.matmul(attn_weights, V)
+        context = context.transpose(1, 2).contiguous().view(batch_size, 1, hidden_dim)
+
+        pooled = self.out_proj(context).squeeze(1)
+        pooled = torch.nan_to_num(pooled, nan=0.0)
+
+        return pooled
+
+
+class MultiHeadBitExtractor(nn.Module):
+    """
+    8 separate extractors for 8 bits - each bit gets its own specialized network.
+    More expressive than single MLP predicting all 8 bits at once.
+    """
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 128):
+        super().__init__()
+
+        self.bit_extractors = nn.ModuleList([
+            nn.Sequential(
+                nn.Linear(hidden_dim, intermediate_dim),
+                nn.GELU(),
+                nn.Linear(intermediate_dim, 1),
+            )
+            for _ in range(8)
+        ])
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        """
+        Args:
+            hidden_states: [batch, hidden_dim]
+
+        Returns:
+            bits: [batch, 8] - one bit from each extractor
+        """
+        hidden_states = torch.nan_to_num(hidden_states, nan=0.0)
+
+        bit_logits = [extractor(hidden_states) for extractor in self.bit_extractors]
+        logits = torch.cat(bit_logits, dim=-1)
+        logits = torch.clamp(logits, -20, 20)
+
+        soft = torch.sigmoid(logits)
+        hard = heaviside_ste(logits)
+        bits = hard - soft.detach() + soft
+
+        return bits, logits
+
+
+class Extractor(nn.Module):
+    """
+    Extracts operands and operation from LLM hidden states.
+    Uses attention pooling and per-bit extraction networks.
+    """
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 256, num_heads: int = 4):
+        super().__init__()
+
+        self.attention_pool = AttentionPooling(hidden_dim, num_heads)
+
+        self.a_extractor = MultiHeadBitExtractor(hidden_dim, intermediate_dim // 2)
+        self.b_extractor = MultiHeadBitExtractor(hidden_dim, intermediate_dim // 2)
+
+        self.op_router = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, len(OPERATIONS)),
+        )
+
+    def forward(self, embeddings: torch.Tensor, mask: torch.Tensor):
+        """
+        Args:
+            embeddings: [batch, seq_len, hidden_dim]
+            mask: [batch, seq_len]
+
+        Returns:
+            a_bits: [batch, 8]
+            b_bits: [batch, 8]
+            op_logits: [batch, 6]
+        """
+        pooled = self.attention_pool(embeddings, mask)
+
+        a_bits, _ = self.a_extractor(pooled)
+        b_bits, _ = self.b_extractor(pooled)
+        op_logits = self.op_router(pooled)
+
+        return a_bits, b_bits, op_logits
+
+
+class PositionExtractor(nn.Module):
+    """
+    Position-specific extraction with dynamic operator detection.
+
+    Tokenization pattern for "A op B":
+        [A_digits...] [operator] [space] [B_digits...]
+
+    Examples:
+        "5 + 3"     -> ['5', ' +', ' ', '3']           (positions: A=0, op=1, B=3)
+        "47 + 86"   -> ['4', '7', ' +', ' ', '8', '6'] (positions: A=0-1, op=2, B=4-5)
+        "127 + 128" -> ['1','2','7',' +', ' ','1','2','8'] (positions: A=0-2, op=3, B=5-7)
+
+    Token IDs (SmolLM2):
+        Digits '0'-'9': 32-41
+        Operators: ' +'=1232, ' -'=731, ' *'=1672, ' >'=2986, ' <'=2067, ' =='=1758
+        Space: 216
+    """
+
+    DIGIT_TOKENS = set(range(32, 42))
+    OPERATOR_TOKENS = {
+        1232: 0,   # ' +' -> add
+        731: 1,    # ' -' -> sub
+        1672: 2,   # ' *' -> mul
+        2986: 3,   # ' >' -> gt
+        2067: 4,   # ' <' -> lt
+        1758: 5,   # ' ==' -> eq
+    }
+    SPACE_TOKEN = 216
+    MAX_DIGITS = 3
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 256):
+        super().__init__()
+        self.hidden_dim = hidden_dim
+
+        self.a_extractor = nn.Sequential(
+            nn.Linear(hidden_dim * self.MAX_DIGITS, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, intermediate_dim // 2),
+            nn.GELU(),
+            nn.Linear(intermediate_dim // 2, 8),
+        )
+
+        self.b_extractor = nn.Sequential(
+            nn.Linear(hidden_dim * self.MAX_DIGITS, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, intermediate_dim // 2),
+            nn.GELU(),
+            nn.Linear(intermediate_dim // 2, 8),
+        )
+
+        self.op_extractor = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim // 2),
+            nn.GELU(),
+            nn.Linear(intermediate_dim // 2, len(OPERATIONS)),
+        )
+
+    def _find_operator_position(self, token_ids: torch.Tensor) -> tuple[int, int]:
+        """
+        Find operator token position and its operation index.
+
+        Args:
+            token_ids: [seq_len] tensor of token IDs
+
+        Returns:
+            (position, op_index) or (-1, -1) if not found
+        """
+        for pos, tid in enumerate(token_ids.tolist()):
+            if tid in self.OPERATOR_TOKENS:
+                return pos, self.OPERATOR_TOKENS[tid]
+        return -1, -1
+
+    def _extract_digit_features(self, hidden: torch.Tensor, start: int, end: int) -> torch.Tensor:
+        """
+        Extract and pad digit hidden states to fixed size.
+
+        Args:
+            hidden: [seq_len, hidden_dim]
+            start: start position (inclusive)
+            end: end position (exclusive)
+
+        Returns:
+            [hidden_dim * MAX_DIGITS] flattened features, zero-padded on the LEFT
+            (so units digit is always at the same position regardless of number length)
+        """
+        n_digits = end - start
+        features = torch.zeros(self.MAX_DIGITS * self.hidden_dim, device=hidden.device)
+
+        if n_digits > 0 and n_digits <= self.MAX_DIGITS:
+            digit_hidden = hidden[start:end, :].reshape(-1)
+            pad_size = (self.MAX_DIGITS - n_digits) * self.hidden_dim
+            features[pad_size:] = digit_hidden
+
+        return features
+
+    def forward(self, hidden: torch.Tensor, mask: torch.Tensor, token_ids: torch.Tensor = None):
+        """
+        Args:
+            hidden: [batch, seq_len, hidden_dim]
+            mask: [batch, seq_len] attention mask
+            token_ids: [batch, seq_len] token IDs (required for operator detection)
+
+        Returns:
+            a_bits: [batch, 8]
+            b_bits: [batch, 8]
+            op_logits: [batch, 6]
+        """
+        if token_ids is None:
+            raise ValueError("PositionExtractor requires token_ids for operator detection")
+
+        batch_size, seq_len, hidden_dim = hidden.shape
+        device = hidden.device
+
+        a_features = []
+        b_features = []
+        op_features = []
+        op_indices = []
+
+        for i in range(batch_size):
+            seq_mask = mask[i].bool()
+            valid_len = seq_mask.sum().item()
+            start_pos = seq_len - valid_len
+
+            valid_tokens = token_ids[i, start_pos:]
+            valid_hidden = hidden[i, start_pos:, :]
+
+            op_pos, op_idx = self._find_operator_position(valid_tokens)
+
+            if op_pos == -1:
+                a_feat = torch.zeros(self.MAX_DIGITS * hidden_dim, device=device)
+                b_feat = torch.zeros(self.MAX_DIGITS * hidden_dim, device=device)
+                op_feat = torch.zeros(hidden_dim, device=device)
+                op_idx = 0
+            else:
+                a_feat = self._extract_digit_features(valid_hidden, 0, op_pos)
+
+                op_feat = valid_hidden[op_pos, :]
+
+                b_start = op_pos + 2 if (op_pos + 1 < valid_len and
+                                          valid_tokens[op_pos + 1].item() == self.SPACE_TOKEN) else op_pos + 1
+                b_feat = self._extract_digit_features(valid_hidden, b_start, valid_len)
+
+            a_features.append(a_feat)
+            b_features.append(b_feat)
+            op_features.append(op_feat)
+            op_indices.append(op_idx)
+
+        a_features = torch.stack(a_features)
+        b_features = torch.stack(b_features)
+        op_features = torch.stack(op_features)
+        op_indices_tensor = torch.tensor(op_indices, device=device, dtype=torch.long)
+
+        a_logits = self.a_extractor(a_features)
+        b_logits = self.b_extractor(b_features)
+        op_logits = self.op_extractor(op_features)
+
+        a_soft = torch.sigmoid(a_logits)
+        b_soft = torch.sigmoid(b_logits)
+        a_hard = heaviside_ste(a_logits)
+        b_hard = heaviside_ste(b_logits)
+        a_bits = a_hard - a_soft.detach() + a_soft
+        b_bits = b_hard - b_soft.detach() + b_soft
+
+        return a_bits, b_bits, op_logits, op_indices_tensor
+
+
+class PositionalDigitExtractor(nn.Module):
+    """
+    Position-aware digit extraction: classifies each digit position independently.
+
+    This approach achieves 100% accuracy because:
+    1. Each digit token position is classified independently (100% accuracy on layer 0)
+    2. Numbers are reconstructed using place values (×100, ×10, ×1)
+    3. No information is lost through pooling
+
+    Token IDs (SmolLM2):
+        Digits '0'-'9': 32-41
+        Operators: ' +'=1232, ' -'=731, ' *'=1672, ' >'=2986, ' <'=2067, ' =='=1758
+        Space: 216
+    """
+
+    DIGIT_TOKENS = set(range(32, 42))
+    OPERATOR_TOKENS = {
+        1232: 0,   # ' +' -> add
+        731: 1,    # ' -' -> sub
+        1672: 2,   # ' *' -> mul
+        2986: 3,   # ' >' -> gt
+        2067: 4,   # ' <' -> lt
+        1758: 5,   # ' ==' -> eq
+    }
+    SPACE_TOKEN = 216
+
+    def __init__(self, hidden_dim: int = 960):
+        super().__init__()
+        self.hidden_dim = hidden_dim
+
+        self.digit_classifier = nn.Linear(hidden_dim, 10)
+
+        self.op_classifier = nn.Linear(hidden_dim, len(OPERATIONS))
+
+    def _find_positions(self, token_ids: torch.Tensor) -> tuple:
+        """Find A digit positions, B digit positions, and operator position."""
+        token_list = token_ids.tolist()
+
+        op_pos = -1
+        op_idx = 0
+        for i, tid in enumerate(token_list):
+            if tid in self.OPERATOR_TOKENS:
+                op_pos = i
+                op_idx = self.OPERATOR_TOKENS[tid]
+                break
+
+        if op_pos == -1:
+            return [], [], -1, 0
+
+        a_positions = [i for i in range(op_pos) if token_list[i] in self.DIGIT_TOKENS]
+
+        b_start = op_pos + 2 if (op_pos + 1 < len(token_list) and
+                                  token_list[op_pos + 1] == self.SPACE_TOKEN) else op_pos + 1
+        b_positions = [i for i in range(b_start, len(token_list)) if token_list[i] in self.DIGIT_TOKENS]
+
+        return a_positions, b_positions, op_pos, op_idx
+
+    def _predict_value(self, hidden: torch.Tensor, positions: list) -> tuple:
+        """Predict digit at each position and reconstruct number."""
+        if not positions:
+            return torch.tensor(0.0, device=hidden.device), []
+
+        digit_logits_list = []
+        soft_value = torch.tensor(0.0, device=hidden.device)
+
+        for idx, pos in enumerate(positions):
+            logits = self.digit_classifier(hidden[pos])
+            digit_logits_list.append(logits)
+
+            probs = torch.softmax(logits, dim=-1)
+            digit_values = torch.arange(10, device=hidden.device, dtype=torch.float32)
+            soft_digit = (probs * digit_values).sum()
+
+            place_value = 10 ** (len(positions) - idx - 1)
+            soft_value = soft_value + soft_digit * place_value
+
+        return soft_value, digit_logits_list
+
+    def _value_to_bits(self, value: torch.Tensor) -> torch.Tensor:
+        """Convert soft value to 8 bits using differentiable operations."""
+        value = torch.clamp(value, 0, 255)
+
+        bits = []
+        for i in range(7, -1, -1):
+            bit = torch.sigmoid((value - (2 ** i - 0.5)) * 10)
+            value = value - bit * (2 ** i)
+            bits.append(bit)
+
+        return torch.stack(bits)
+
+    def forward(self, hidden: torch.Tensor, mask: torch.Tensor, token_ids: torch.Tensor):
+        """
+        Args:
+            hidden: [batch, seq_len, hidden_dim]
+            mask: [batch, seq_len]
+            token_ids: [batch, seq_len]
+
+        Returns:
+            a_bits: [batch, 8]
+            b_bits: [batch, 8]
+            op_logits: [batch, 6]
+            op_indices: [batch] ground truth op from tokens
+            a_digit_logits: list of [batch, 10] per digit position
+            b_digit_logits: list of [batch, 10] per digit position
+        """
+        batch_size = hidden.shape[0]
+        device = hidden.device
+
+        a_bits_list = []
+        b_bits_list = []
+        op_logits_list = []
+        op_indices_list = []
+        a_values_list = []
+        b_values_list = []
+        a_digit_logits_list = []
+        b_digit_logits_list = []
+
+        for i in range(batch_size):
+            seq_mask = mask[i].bool()
+            valid_len = seq_mask.sum().item()
+            start_pos = hidden.shape[1] - valid_len
+
+            valid_hidden = hidden[i, start_pos:]
+            valid_tokens = token_ids[i, start_pos:]
+
+            a_pos, b_pos, op_pos, op_idx = self._find_positions(valid_tokens)
+
+            a_value, a_digit_logits = self._predict_value(valid_hidden, a_pos)
+            b_value, b_digit_logits = self._predict_value(valid_hidden, b_pos)
+
+            a_bits = self._value_to_bits(a_value)
+            b_bits = self._value_to_bits(b_value)
+
+            if op_pos >= 0:
+                op_logits = self.op_classifier(valid_hidden[op_pos])
+            else:
+                op_logits = torch.zeros(len(OPERATIONS), device=device)
+
+            a_bits_list.append(a_bits)
+            b_bits_list.append(b_bits)
+            op_logits_list.append(op_logits)
+            op_indices_list.append(op_idx)
+            a_values_list.append(a_value)
+            b_values_list.append(b_value)
+            a_digit_logits_list.append(a_digit_logits)
+            b_digit_logits_list.append(b_digit_logits)
+
+        a_bits = torch.stack(a_bits_list)
+        b_bits = torch.stack(b_bits_list)
+        op_logits = torch.stack(op_logits_list)
+        op_indices = torch.tensor(op_indices_list, device=device, dtype=torch.long)
+        a_values = torch.stack(a_values_list)
+        b_values = torch.stack(b_values_list)
+
+        return a_bits, b_bits, op_logits, op_indices, a_values, b_values, a_digit_logits_list, b_digit_logits_list
+
+
+class DigitExtractor(nn.Module):
+    """
+    Digit-level extraction: predicts digits (0-9) then converts to bits.
+    Uses attention pooling (less accurate than PositionalDigitExtractor).
+    """
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 256, num_heads: int = 4):
+        super().__init__()
+
+        self.attention_pool = AttentionPooling(hidden_dim, num_heads)
+
+        self.a_digit_pred = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, 3 * 10),
+        )
+
+        self.b_digit_pred = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, 3 * 10),
+        )
+
+        self.op_router = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Linear(intermediate_dim, len(OPERATIONS)),
+        )
+
+    def digits_to_bits(self, digit_logits: torch.Tensor) -> torch.Tensor:
+        """
+        Convert 3-digit predictions to 8-bit representation.
+        digit_logits: [batch, 30] (3 digits * 10 classes each)
+        Returns: [batch, 8] bits
+        """
+        batch_size = digit_logits.shape[0]
+
+        logits = digit_logits.view(batch_size, 3, 10)
+        probs = torch.softmax(logits, dim=-1)
+
+        digit_values = torch.arange(10, device=digit_logits.device).float()
+        soft_digits = (probs * digit_values).sum(dim=-1)
+
+        hundreds = soft_digits[:, 0]
+        tens = soft_digits[:, 1]
+        ones = soft_digits[:, 2]
+
+        value = hundreds * 100 + tens * 10 + ones
+        value = torch.clamp(value, 0, 255)
+
+        bits = []
+        for i in range(7, -1, -1):
+            bit = torch.fmod(torch.floor(value / (2 ** i)), 2)
+            bits.append(bit)
+
+        return torch.stack(bits, dim=-1)
+
+    def forward(self, hidden: torch.Tensor, mask: torch.Tensor):
+        """
+        Returns:
+            a_bits, b_bits, op_logits, a_digit_logits, b_digit_logits
+        """
+        pooled = self.attention_pool(hidden, mask)
+
+        a_digit_logits = self.a_digit_pred(pooled)
+        b_digit_logits = self.b_digit_pred(pooled)
+        op_logits = self.op_router(pooled)
+
+        a_bits = self.digits_to_bits(a_digit_logits)
+        b_bits = self.digits_to_bits(b_digit_logits)
+
+        return a_bits, b_bits, op_logits, a_digit_logits, b_digit_logits
+
+
+class HybridExtractor(nn.Module):
+    """
+    Hybrid extractor that handles both digit tokens and word numbers.
+
+    For digit tokens (32-41): Direct lookup, no training needed
+    For word numbers: Learned MLP extraction from pooled hidden states
+
+    This is the real training target - learning to extract numbers from
+    natural language like "forty seven plus eighty six".
+    """
+
+    DIGIT_TOKENS = set(range(32, 42))
+    SYMBOL_OP_TOKENS = {
+        1232: 0,   # ' +' -> add
+        731: 1,    # ' -' -> sub
+        1672: 2,   # ' *' -> mul
+        2986: 3,   # ' >' -> gt
+        2067: 4,   # ' <' -> lt
+        1758: 5,   # ' ==' -> eq
+    }
+    WORD_OP_TOKENS = {
+        2068: 0,   # 'plus' -> add
+        8500: 1,   # 'minus' -> sub
+        1580: 2,   # 'times' -> mul
+        6301: 3,   # 'greater' -> gt
+        1912: 4,   # 'less' -> lt
+        16364: 5,  # 'equals' -> eq
+        11540: 5,  # 'equal' -> eq
+    }
+    ALL_OP_TOKENS = {**SYMBOL_OP_TOKENS, **WORD_OP_TOKENS}
+
+    def __init__(self, hidden_dim: int = 960, intermediate_dim: int = 256, num_heads: int = 4):
+        super().__init__()
+        self.hidden_dim = hidden_dim
+
+        self.attention_pool = AttentionPooling(hidden_dim, num_heads)
+        self.a_pool = AttentionPooling(hidden_dim, num_heads)
+        self.b_pool = AttentionPooling(hidden_dim, num_heads)
+
+        self.a_digit_pred = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Dropout(0.1),
+            nn.Linear(intermediate_dim, 3 * 10),
+        )
+
+        self.b_digit_pred = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim),
+            nn.GELU(),
+            nn.Dropout(0.1),
+            nn.Linear(intermediate_dim, 3 * 10),
+        )
+
+        self.op_predictor = nn.Sequential(
+            nn.Linear(hidden_dim, intermediate_dim // 2),
+            nn.GELU(),
+            nn.Linear(intermediate_dim // 2, len(OPERATIONS)),
+        )
+
+    def _has_digit_tokens(self, token_ids: torch.Tensor) -> bool:
+        """Check if input contains digit tokens."""
+        for tid in token_ids.tolist():
+            if tid in self.DIGIT_TOKENS:
+                return True
+        return False
+
+    def _find_op_position(self, token_ids: torch.Tensor) -> int:
+        """Find position of operator token, returns -1 if not found."""
+        tokens = token_ids.tolist()
+        for i, tid in enumerate(tokens):
+            if tid in self.ALL_OP_TOKENS:
+                return i
+        return -1
+
+    def _extract_from_digits(self, token_ids: torch.Tensor) -> tuple:
+        """
+        Extract values directly from digit tokens (hardcoded lookup).
+        Handles both symbol operators (' +') and word operators ('plus').
+        Returns (a_value, b_value, op_idx) or None if pattern not found.
+        """
+        tokens = token_ids.tolist()
+
+        op_pos = -1
+        op_idx = 0
+        for i, tid in enumerate(tokens):
+            if tid in self.ALL_OP_TOKENS:
+                op_pos = i
+                op_idx = self.ALL_OP_TOKENS[tid]
+                break
+
+        if op_pos == -1:
+            return None
+
+        a_digits = []
+        for i in range(op_pos):
+            if tokens[i] in self.DIGIT_TOKENS:
+                a_digits.append(tokens[i] - 32)
+
+        b_start = op_pos + 1
+        if b_start < len(tokens) and tokens[b_start] == 216:
+            b_start += 1
+
+        b_digits = []
+        for i in range(b_start, len(tokens)):
+            if tokens[i] in self.DIGIT_TOKENS:
+                b_digits.append(tokens[i] - 32)
+
+        if not a_digits or not b_digits:
+            return None
+
+        a_val = 0
+        for d in a_digits:
+            a_val = a_val * 10 + d
+
+        b_val = 0
+        for d in b_digits:
+            b_val = b_val * 10 + d
+
+        return min(a_val, 255), min(b_val, 255), op_idx
+
+    def _value_to_bits(self, value: int, device) -> torch.Tensor:
+        """Convert integer to 8-bit tensor."""
+        bits = torch.zeros(8, device=device)
+        for i in range(8):
+            bits[7 - i] = (value >> i) & 1
+        return bits
+
+    def _digits_to_value_and_bits(self, digit_logits: torch.Tensor, device) -> tuple:
+        """
+        Convert 3-digit logits to value and bits.
+        digit_logits: [30] (3 digits × 10 classes)
+        Returns: (value tensor, bits tensor [8])
+        """
+        logits = digit_logits.view(3, 10)
+        probs = torch.softmax(logits, dim=-1)
+
+        digit_values = torch.arange(10, device=device, dtype=torch.float32)
+        soft_digits = (probs * digit_values).sum(dim=-1)
+
+        hundreds = soft_digits[0]
+        tens = soft_digits[1]
+        ones = soft_digits[2]
+
+        value = hundreds * 100 + tens * 10 + ones
+        value = torch.clamp(value, 0, 255)
+
+        bits = []
+        for i in range(7, -1, -1):
+            threshold = 2 ** i
+            bit = torch.sigmoid((value - threshold + 0.5) * 10)
+            bits.append(bit)
+            value = value - bit * threshold
+        return hundreds * 100 + tens * 10 + ones, torch.stack(bits)
+
+    def forward(self, hidden: torch.Tensor, mask: torch.Tensor, token_ids: torch.Tensor = None):
+        """
+        Args:
+            hidden: [batch, seq_len, hidden_dim]
+            mask: [batch, seq_len]
+            token_ids: [batch, seq_len] - optional, enables digit lookup
+
+        Returns:
+            a_bits, b_bits, op_logits, a_values, b_values, used_lookup,
+            a_digit_logits, b_digit_logits
+        """
+        batch_size = hidden.shape[0]
+        device = hidden.device
+
+        a_bits_list = []
+        a_digit_logits_list = []
+        b_digit_logits_list = []
+        b_bits_list = []
+        op_logits_list = []
+        a_values_list = []
+        b_values_list = []
+        used_lookup_list = []
+
+        pooled = self.attention_pool(hidden, mask)
+
+        for i in range(batch_size):
+            lookup_result = None
+            if token_ids is not None:
+                seq_mask = mask[i].bool()
+                valid_len = seq_mask.sum().item()
+                start_pos = hidden.shape[1] - valid_len
+                valid_tokens = token_ids[i, start_pos:]
+
+                if self._has_digit_tokens(valid_tokens):
+                    lookup_result = self._extract_from_digits(valid_tokens)
+
+            if lookup_result is not None:
+                a_val, b_val, op_idx = lookup_result
+                a_bits = self._value_to_bits(a_val, device)
+                b_bits = self._value_to_bits(b_val, device)
+                op_logits = torch.zeros(len(OPERATIONS), device=device)
+                op_logits[op_idx] = 10.0
+
+                a_bits_list.append(a_bits)
+                b_bits_list.append(b_bits)
+                op_logits_list.append(op_logits)
+                a_values_list.append(float(a_val))
+                b_values_list.append(float(b_val))
+                used_lookup_list.append(True)
+                a_digit_logits_list.append(None)
+                b_digit_logits_list.append(None)
+            else:
+                sample_hidden = hidden[i:i+1]
+                sample_mask = mask[i:i+1]
+
+                seq_mask = mask[i].bool()
+                valid_len = int(seq_mask.sum().item())
+                start_pos = hidden.shape[1] - valid_len
+                valid_tokens = token_ids[i, start_pos:] if token_ids is not None else None
+
+                op_pos = self._find_op_position(valid_tokens) if valid_tokens is not None else -1
+
+                if op_pos > 0 and op_pos < valid_len - 1:
+                    a_end = start_pos + op_pos
+                    b_start = start_pos + op_pos + 1
+
+                    a_mask = torch.zeros_like(sample_mask)
+                    a_mask[0, start_pos:a_end] = 1.0
+                    b_mask = torch.zeros_like(sample_mask)
+                    b_mask[0, b_start:] = sample_mask[0, b_start:]
+
+                    a_pooled = self.a_pool(sample_hidden, a_mask)[0]
+                    b_pooled = self.b_pool(sample_hidden, b_mask)[0]
+                else:
+                    a_pooled = pooled[i]
+                    b_pooled = pooled[i]
+
+                a_digit_logits = self.a_digit_pred(a_pooled)
+                b_digit_logits = self.b_digit_pred(b_pooled)
+                op_logits = self.op_predictor(pooled[i])
+
+                a_val, a_bits = self._digits_to_value_and_bits(a_digit_logits, device)
+                b_val, b_bits = self._digits_to_value_and_bits(b_digit_logits, device)
+
+                a_bits_list.append(a_bits)
+                b_bits_list.append(b_bits)
+                op_logits_list.append(op_logits)
+                a_values_list.append(a_val)
+                b_values_list.append(b_val)
+                used_lookup_list.append(False)
+                a_digit_logits_list.append(a_digit_logits)
+                b_digit_logits_list.append(b_digit_logits)
+
+        a_bits = torch.stack(a_bits_list)
+        b_bits = torch.stack(b_bits_list)
+        op_logits = torch.stack(op_logits_list)
+        a_values = torch.stack([v if isinstance(v, torch.Tensor) else torch.tensor(v, device=device) for v in a_values_list])
+        b_values = torch.stack([v if isinstance(v, torch.Tensor) else torch.tensor(v, device=device) for v in b_values_list])
+        used_lookup = torch.tensor(used_lookup_list, device=device, dtype=torch.bool)
+        valid_a_logits = [x for x in a_digit_logits_list if x is not None]
+        valid_b_logits = [x for x in b_digit_logits_list if x is not None]
+        a_digit_logits_out = torch.stack(valid_a_logits) if valid_a_logits else None
+        b_digit_logits_out = torch.stack(valid_b_logits) if valid_b_logits else None
+
+        return a_bits, b_bits, op_logits, a_values, b_values, used_lookup, a_digit_logits_out, b_digit_logits_out
+
+    def _soft_value_to_bits(self, value: torch.Tensor, device) -> torch.Tensor:
+        """Convert soft value (0-255) to 8-bit representation differentiably."""
+        value = torch.clamp(value, 0, 255)
+        bits = []
+        remaining = value
+        for i in range(7, -1, -1):
+            threshold = 2 ** i
+            bit = torch.sigmoid((remaining - threshold + 0.5) * 10)
+            bits.append(bit)
+            remaining = remaining - bit * threshold
+        return torch.stack(bits)
+
+
+class ArithmeticModel(nn.Module):
+    """
+    LLM + extractor + frozen threshold circuits.
+    Optionally unfreeze top N transformer layers with --unfreeze_layers.
+    """
+
+    def __init__(self, device: str = 'cuda', unfreeze_layers: int = 0,
+                 extract_layer: int = -1, position_extract: bool = False,
+                 digit_pred: bool = False, positional_digit: bool = False,
+                 hybrid: bool = False):
+        super().__init__()
+        self.device = device
+        self.unfreeze_layers = unfreeze_layers
+        self.extract_layer = extract_layer
+        self.position_extract = position_extract
+        self.digit_pred = digit_pred
+        self.positional_digit = positional_digit
+        self.hybrid = hybrid
+
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+
+        print("[1/4] Loading tokenizer...", flush=True)
+        self.tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+        self.tokenizer.padding_side = 'left'
+        if self.tokenizer.pad_token is None:
+            self.tokenizer.pad_token = self.tokenizer.eos_token
+        print("  Tokenizer loaded.", flush=True)
+
+        print("[2/4] Loading SmolLM2-360M...", flush=True)
+        self.llm = AutoModelForCausalLM.from_pretrained(
+            MODEL_ID,
+            torch_dtype=torch.float16,
+            device_map=device,
+            output_hidden_states=True
+        )
+
+        for param in self.llm.parameters():
+            param.requires_grad = False
+
+        if unfreeze_layers > 0:
+            num_layers = len(self.llm.model.layers)
+            layers_to_unfreeze = list(range(num_layers - unfreeze_layers, num_layers))
+            print(f"  Unfreezing layers {layers_to_unfreeze}...", flush=True)
+            for layer_idx in layers_to_unfreeze:
+                for param in self.llm.model.layers[layer_idx].parameters():
+                    param.requires_grad = True
+
+        hidden_dim = self.llm.config.hidden_size
+        llm_params = sum(p.numel() for p in self.llm.parameters())
+        trainable_llm = sum(p.numel() for p in self.llm.parameters() if p.requires_grad)
+        print(f"  LLM loaded. Hidden dim: {hidden_dim}", flush=True)
+        print(f"  LLM params: {llm_params:,} total, {trainable_llm:,} trainable", flush=True)
+
+        print("[3/4] Loading threshold circuits...", flush=True)
+        self.circuits = FrozenThresholdCircuits(device=device)
+        print(f"  Circuits loaded. {len(self.circuits.weights)} tensors", flush=True)
+
+        print("[4/4] Initializing extractor...", flush=True)
+        if hybrid:
+            print("  Using HYBRID extraction (digit lookup + word learning)", flush=True)
+            self.extractor = HybridExtractor(
+                hidden_dim=hidden_dim,
+                intermediate_dim=256,
+                num_heads=4
+            ).to(device)
+        elif positional_digit:
+            print("  Using POSITIONAL DIGIT extraction (100% proven)", flush=True)
+            self.extractor = PositionalDigitExtractor(
+                hidden_dim=hidden_dim
+            ).to(device)
+        elif position_extract:
+            print("  Using position-specific extraction", flush=True)
+            self.extractor = PositionExtractor(
+                hidden_dim=hidden_dim,
+                intermediate_dim=256
+            ).to(device)
+        elif digit_pred:
+            print("  Using digit-level prediction", flush=True)
+            self.extractor = DigitExtractor(
+                hidden_dim=hidden_dim,
+                intermediate_dim=256,
+                num_heads=4
+            ).to(device)
+        else:
+            self.extractor = Extractor(
+                hidden_dim=hidden_dim,
+                intermediate_dim=256,
+                num_heads=4
+            ).to(device)
+
+        if extract_layer != -1:
+            print(f"  Extracting from layer {extract_layer}", flush=True)
+
+        trainable_ext = sum(p.numel() for p in self.extractor.parameters())
+        total_trainable = trainable_llm + trainable_ext
+        print(f"  Extractor params: {trainable_ext:,}", flush=True)
+        print(f"  Total trainable: {total_trainable:,}", flush=True)
+
+    def get_hidden_states(self, texts: list[str]) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
+        """
+        Get hidden states from specified layer.
+
+        Returns:
+            hidden: [batch, seq_len, hidden_dim] hidden states
+            mask: [batch, seq_len] attention mask
+            token_ids: [batch, seq_len] input token IDs
+        """
+        inputs = self.tokenizer(
+            texts,
+            return_tensors='pt',
+            padding=True,
+            truncation=True,
+            max_length=64
+        ).to(self.device)
+
+        if self.unfreeze_layers > 0:
+            outputs = self.llm(**inputs, output_hidden_states=True)
+        else:
+            with torch.no_grad():
+                outputs = self.llm(**inputs, output_hidden_states=True)
+
+        hidden = outputs.hidden_states[self.extract_layer].float()
+        mask = inputs.attention_mask.float()
+        token_ids = inputs.input_ids
+
+        return hidden, mask, token_ids
+
+    def forward(self, texts: list[str]):
+        """
+        Full forward pass: text -> hidden states -> extractor -> circuits -> result
+
+        Returns:
+            result_bits, a_bits, b_bits, op_logits
+            If digit_pred: also returns a_digit_logits, b_digit_logits
+            If position_extract or positional_digit: also returns op_indices
+            If positional_digit: also returns a_values, b_values
+        """
+        hidden, mask, token_ids = self.get_hidden_states(texts)
+
+        if self.hybrid or self.positional_digit or self.position_extract:
+            extractor_out = self.extractor(hidden, mask, token_ids)
+        else:
+            extractor_out = self.extractor(hidden, mask)
+
+        if self.hybrid:
+            a_bits, b_bits, op_logits, a_values, b_values, used_lookup, a_digit_logits, b_digit_logits = extractor_out
+            op_indices_from_tokens = None
+        elif self.positional_digit:
+            a_bits, b_bits, op_logits, op_indices_from_tokens, a_values, b_values, a_digit_logits, b_digit_logits = extractor_out
+            used_lookup = None
+        elif self.digit_pred:
+            a_bits, b_bits, op_logits, a_digit_logits, b_digit_logits = extractor_out
+            op_indices_from_tokens = None
+            a_values, b_values = None, None
+            used_lookup = None
+        elif self.position_extract:
+            a_bits, b_bits, op_logits, op_indices_from_tokens = extractor_out
+            a_digit_logits, b_digit_logits = None, None
+            a_values, b_values = None, None
+            used_lookup = None
+        else:
+            a_bits, b_bits, op_logits = extractor_out
+            a_digit_logits, b_digit_logits = None, None
+            op_indices_from_tokens = None
+            a_values, b_values = None, None
+            used_lookup = None
+
+        op_probs = torch.softmax(op_logits, dim=-1)
+
+        result_bits = self.circuits(a_bits, b_bits, op_probs)
+
+        if self.hybrid:
+            return result_bits, a_bits, b_bits, op_logits, a_values, b_values, used_lookup, a_digit_logits, b_digit_logits
+        if self.positional_digit:
+            return result_bits, a_bits, b_bits, op_logits, op_indices_from_tokens, a_values, b_values, a_digit_logits, b_digit_logits
+        if self.digit_pred:
+            return result_bits, a_bits, b_bits, op_logits, a_digit_logits, b_digit_logits
+        if self.position_extract:
+            return result_bits, a_bits, b_bits, op_logits, op_indices_from_tokens
+        return result_bits, a_bits, b_bits, op_logits
+
+    def trainable_parameters(self):
+        """Return all trainable parameters for optimizer."""
+        params = list(self.extractor.parameters())
+        if self.unfreeze_layers > 0:
+            params += [p for p in self.llm.parameters() if p.requires_grad]
+        return params
+
+
+if __name__ == "__main__":
+    import sys
+    sys.path.insert(0, '.')
+    from fitness import generate_batch, compute_fitness, OPERATIONS
+
+    print("Testing model components...")
+
+    device = 'cuda'
+    batch = generate_batch(32, device)
+
+    print("\n1. Testing DirectCircuitModel (should get ~100% fitness)...")
+    direct_model = DirectCircuitModel(device=device)
+
+    def direct_fn(a, b, op):
+        return direct_model(a, b, op)
+
+    fitness, details = compute_fitness(direct_fn, n_samples=2000, batch_size=128,
+                                       device=device, return_details=True)
+    print(f"   Direct circuit fitness: {fitness:.4f}")
+    for op in OPERATIONS:
+        acc = details['by_op'][op]['accuracy']
+        print(f"   {op}: {acc:.4f}")
+
+    print("\n2. Testing ThresholdALU (trainable interface)...")
+    model = ThresholdALU(device=device)
+
+    x = torch.cat([batch['a_bits'], batch['b_bits'], batch['op_onehot']], dim=-1)
+    a_enc, b_enc = model.encoder(x)
+    print(f"   Encoder output shapes: a={a_enc.shape}, b={b_enc.shape}")
+
+    op_weights = model.router(x)
+    print(f"   Router output shape: {op_weights.shape}")
+    print(f"   Router output sample: {op_weights[0].tolist()}")
+
+    result = model(batch['a_bits'], batch['b_bits'], batch['op_onehot'])
+    print(f"   Full model output shape: {result.shape}")
+
+    print("\n3. Testing untrained ThresholdALU fitness...")
+
+    def model_fn(a, b, op):
+        return model(a, b, op)
+
+    fitness = compute_fitness(model_fn, n_samples=1000, batch_size=128, device=device)
+    print(f"   Untrained model fitness: {fitness:.4f} (expected low)")
+
+    print("\n4. Counting parameters...")
+    total = sum(p.numel() for p in model.parameters() if p.requires_grad)
+    encoder_params = sum(p.numel() for p in model.encoder.parameters())
+    router_params = sum(p.numel() for p in model.router.parameters())
+    print(f"   Encoder: {encoder_params:,}")
+    print(f"   Router: {router_params:,}")
+    print(f"   Total trainable: {total:,}")
+
+    print("\nDone.")

llm_integration/smollm2/SMOLLM2_ARCHITECTURE.md

@@ -0,0 +1,456 @@
+# SmolLM2-360M-Instruct Architecture Analysis
+
+Technical reference document for the 8bit-threshold-computer LLM integration project.
+
+**Model**: `HuggingFaceTB/SmolLM2-360M-Instruct`
+**Architecture**: LlamaForCausalLM (Llama 2 variant)
+**Tokenizer**: GPT2TokenizerFast
+**Analysis Date**: 2026-01-21
+
+---
+
+## 1. Executive Summary
+
+SmolLM2-360M-Instruct is a 362M parameter causal language model using the Llama architecture. Key characteristics relevant to our bit extraction task:
+
+- **Hidden dimension: 960** (matches our extractor input requirement)
+- **32 transformer layers** providing multiple extraction points
+- **Digit-level tokenization** for numbers (each digit is a separate token)
+- **Grouped Query Attention (GQA)** with 15 query heads and 5 KV heads
+
+---
+
+## 2. Architecture Census
+
+### 2.1 Core Parameters
+
+| Parameter | Value |
+|-----------|-------|
+| Total Parameters | 361,821,120 (361.82M) |
+| Vocabulary Size | 49,152 |
+| Hidden Dimension | 960 |
+| Intermediate Dimension (MLP) | 2,560 |
+| Number of Layers | 32 |
+| Number of Attention Heads | 15 |
+| Number of KV Heads | 5 (Grouped Query Attention) |
+| Head Dimension | 64 |
+| Max Sequence Length | 8,192 |
+| Activation Function | SiLU |
+| Normalization | RMSNorm (eps=1e-05) |
+| Position Encoding | RoPE (theta=100,000) |
+| Word Embedding Tying | Yes (embed_tokens = lm_head) |
+
+### 2.2 Architecture Diagram
+
+```
+Input Token IDs
+       |
+       v
++------------------+
+| Embedding Layer  |  (49152, 960)
++------------------+
+       |
+       v
++------------------+
+| LlamaDecoderLayer| x 32
+|  +-------------+ |
+|  | RMSNorm     | |
+|  +-------------+ |
+|  | Self-Attn   | |  Q: (960, 960), K: (960, 320), V: (960, 320), O: (960, 960)
+|  +-------------+ |
+|  | Residual    | |
+|  +-------------+ |
+|  | RMSNorm     | |
+|  +-------------+ |
+|  | MLP (SwiGLU)| |  gate: (960, 2560), up: (960, 2560), down: (2560, 960)
+|  +-------------+ |
+|  | Residual    | |
++------------------+
+       |
+       v
++------------------+
+| Final RMSNorm    |  (960,)
++------------------+
+       |
+       v
++------------------+
+| LM Head          |  (960, 49152) - tied with embeddings
++------------------+
+       |
+       v
+Logits (batch, seq, 49152)
+```
+
+### 2.3 Parameter Distribution
+
+| Component | Parameters | Percentage |
+|-----------|-----------|------------|
+| Embedding | 47,185,920 | 13.04% |
+| All Attention Layers | 78,643,200 | 21.74% |
+| All MLP Layers | 235,929,600 | 65.19% |
+| All Layer Norms | 61,440 | 0.02% |
+| Final Norm | 960 | 0.00% |
+
+Per-layer breakdown (each of 32 layers):
+- Attention: 2,457,600 params (0.68%)
+- MLP: 7,372,800 params (2.04%)
+- Norms: 1,920 params (0.00%)
+
+---
+
+## 3. Weight Inventory
+
+### 3.1 Embedding and Output Layers
+
+| Parameter Name | Shape | Elements | Notes |
+|---------------|-------|----------|-------|
+| `model.embed_tokens.weight` | (49152, 960) | 47,185,920 | Token embeddings |
+| `model.norm.weight` | (960,) | 960 | Final layer norm |
+| `lm_head.weight` | (49152, 960) | (tied) | Tied to embed_tokens |
+
+### 3.2 Single Transformer Layer Structure
+
+Each of the 32 layers (`model.layers.{0-31}`) contains:
+
+**Attention Block:**
+| Parameter | Shape | Elements |
+|-----------|-------|----------|
+| `self_attn.q_proj.weight` | (960, 960) | 921,600 |
+| `self_attn.k_proj.weight` | (320, 960) | 307,200 |
+| `self_attn.v_proj.weight` | (320, 960) | 307,200 |
+| `self_attn.o_proj.weight` | (960, 960) | 921,600 |
+| **Attention Total** | | **2,457,600** |
+
+**MLP Block (SwiGLU):**
+| Parameter | Shape | Elements |
+|-----------|-------|----------|
+| `mlp.gate_proj.weight` | (2560, 960) | 2,457,600 |
+| `mlp.up_proj.weight` | (2560, 960) | 2,457,600 |
+| `mlp.down_proj.weight` | (960, 2560) | 2,457,600 |
+| **MLP Total** | | **7,372,800** |
+
+**Normalization:**
+| Parameter | Shape | Elements |
+|-----------|-------|----------|
+| `input_layernorm.weight` | (960,) | 960 |
+| `post_attention_layernorm.weight` | (960,) | 960 |
+| **Norms Total** | | **1,920** |
+
+**Layer Total: 9,832,320 parameters**
+
+### 3.3 Grouped Query Attention (GQA) Details
+
+SmolLM2 uses GQA with a 3:1 ratio:
+- 15 query heads (Q dimension: 960 = 15 x 64)
+- 5 key-value heads (KV dimension: 320 = 5 x 64)
+- Each KV head is shared by 3 query heads
+- This reduces KV cache memory by ~67% vs standard MHA
+
+---
+
+## 4. Tokenization Analysis
+
+### 4.1 Arithmetic Expression Tokenization
+
+Test input: `"47 + 86"`
+
+| Position | Token ID | Token | Description |
+|----------|----------|-------|-------------|
+| 0 | 36 | `'4'` | First digit of operand A |
+| 1 | 39 | `'7'` | Second digit of operand A |
+| 2 | 1232 | `' +'` | Space + plus sign |
+| 3 | 216 | `' '` | Trailing space |
+| 4 | 40 | `'8'` | First digit of operand B |
+| 5 | 38 | `'6'` | Second digit of operand B |
+
+### 4.2 Digit Token Mappings
+
+| Digit | Token ID |
+|-------|----------|
+| 0 | 32 |
+| 1 | 33 |
+| 2 | 34 |
+| 3 | 35 |
+| 4 | 36 |
+| 5 | 37 |
+| 6 | 38 |
+| 7 | 39 |
+| 8 | 40 |
+| 9 | 41 |
+
+Key observations:
+- **Digits are tokenized individually** (no multi-digit tokens like "47")
+- Digit tokens are sequential: ID = 32 + digit_value
+- Space-prefixed operators exist (e.g., `' +'` = 1232)
+- `'='` has token ID 45
+
+### 4.3 Implications for Bit Extraction
+
+The digit-by-digit tokenization means:
+1. For `"47 + 86"`, operand A spans positions [0,1] and operand B spans positions [4,5]
+2. The model must learn to:
+   - Recognize digit boundaries
+   - Compose multi-digit numbers from sequential tokens
+   - Perform arithmetic across token positions
+3. Hidden states at digit positions contain the numerical representation
+
+---
+
+## 5. Hidden State Analysis
+
+### 5.1 Hidden State Output Structure
+
+When running with `output_hidden_states=True`:
+- Returns **33 hidden states** (embedding + 32 layer outputs)
+- Each has shape: `(batch_size, seq_len, hidden_dim)`
+- For `"47 + 86"`: `(1, 6, 960)`
+
+### 5.2 Hidden State Statistics by Layer
+
+| Layer | Mean | Std Dev | Min | Max |
+|-------|------|---------|-----|-----|
+| Embedding | -0.001 | 0.105 | -0.44 | 1.77 |
+| Layer 0 | -0.127 | 2.55 | -80.8 | 19.0 |
+| Layer 1 | -0.171 | 3.70 | -161 | 39.7 |
+| Layer 2 | -0.151 | 3.67 | -102 | 61.4 |
+| Layer 3 | -1.13 | 327 | -21,722 | 11,856 |
+| Layer 4-12 | ~-1.3 | ~327 | ~-21,700 | ~11,800 |
+| Layer 13-26 | ~-1.5 | ~337 | ~-22,400 | ~12,100 |
+| Layer 27-30 | ~-1.8 | ~310 | ~-20,000 | ~11,800 |
+| Layer 31 | 0.017 | 1.34 | -18.9 | 34.3 |
+
+Key observations:
+1. **Dramatic variance explosion at Layer 3**: Std dev jumps from ~4 to ~327
+2. **Stable middle layers (4-26)**: Consistent statistics, suggesting numerical computation
+3. **Compression at final layer**: Std dev drops to 1.34 at Layer 31 (pre-softmax normalization)
+4. **Layer 31 is well-scaled** for downstream processing
+
+### 5.3 Extraction Point Candidates
+
+| Layer Range | Characteristics | Suitability |
+|-------------|-----------------|-------------|
+| 0-2 (Early) | Low variance, close to embeddings | Poor - minimal computation |
+| 3-12 (Early-Mid) | High variance, initial processing | Moderate - may contain raw numerical features |
+| 13-26 (Middle) | Stable high variance | Good - computation in progress |
+| 27-30 (Late) | Variance compression begins | Good - refined representations |
+| 31 (Final) | Well-normalized output | Best - final representation before LM head |
+
+---
+
+## 6. Relevance to 8bit-Threshold-Computer Project
+
+### 6.1 Hidden Dimension Match
+
+**The hidden dimension of 960 exactly matches our extractor input requirement.** This is fortuitous as it means:
+- No projection layer needed to interface with our bit extractor
+- Direct extraction from any layer's hidden states
+- Full utilization of the model's representational capacity
+
+### 6.2 Recommended Extraction Strategy
+
+```python
+def extract_hidden_state(model, tokenizer, expression, layer=-1):
+    """
+    Extract hidden state for bit extraction.
+
+    Args:
+        layer: Which layer to extract from (default: final layer)
+               -1 = Layer 31 (final, pre-LM-head)
+
+    Returns:
+        Tensor of shape (960,) for the last token position
+    """
+    inputs = tokenizer(expression, return_tensors="pt")
+    outputs = model(**inputs, output_hidden_states=True)
+
+    # hidden_states[0] = embedding, hidden_states[1] = layer 0, ...
+    # hidden_states[32] = layer 31 (final)
+    hidden = outputs.hidden_states[layer]  # (1, seq_len, 960)
+
+    # Extract last token position for autoregressive prediction
+    return hidden[0, -1, :]  # (960,)
+```
+
+### 6.3 Token Position Analysis
+
+For arithmetic expressions like `"A + B"`:
+
+```
+Tokens:    [d1] [d2] [ +] [ ] [d3] [d4]
+Positions:  0    1    2   3   4    5
+
+Operand A: positions 0 to (plus_pos - 1)
+Operator:  position where ' +' token appears
+Operand B: positions (plus_pos + 2) to end
+```
+
+Strategy for operand extraction:
+1. Find the `' +'` token (ID 1232) position
+2. Collect hidden states at digit positions before it (operand A)
+3. Collect hidden states at digit positions after it (operand B)
+4. Consider pooling (mean, max) or concatenating digit hidden states
+
+### 6.4 Attention Pattern Utilization
+
+With GQA (15 query heads, 5 KV heads), we can analyze attention patterns to:
+1. Identify which positions attend to operand digits
+2. Determine if the model explicitly links corresponding digit positions
+3. Find heads that specialize in numerical reasoning
+
+```python
+def get_attention_weights(model, tokenizer, expression):
+    inputs = tokenizer(expression, return_tensors="pt")
+    outputs = model(**inputs, output_attentions=True)
+    # attentions: tuple of (batch, num_heads, seq_len, seq_len) per layer
+    return outputs.attentions
+```
+
+### 6.5 Extraction Interface Specification
+
+For integration with the threshold computer:
+
+```python
+class SmolLM2Extractor:
+    """Interface between SmolLM2 and threshold-based bit extraction."""
+
+    def __init__(self, model, tokenizer, extraction_layer=31):
+        self.model = model
+        self.tokenizer = tokenizer
+        self.layer = extraction_layer + 1  # +1 because index 0 is embedding
+
+    def get_hidden_state(self, text: str) -> torch.Tensor:
+        """
+        Returns: Tensor of shape (960,) ready for bit extractor
+        """
+        tokens = self.tokenizer(text, return_tensors="pt")
+        with torch.no_grad():
+            outputs = self.model(**tokens, output_hidden_states=True)
+        return outputs.hidden_states[self.layer][0, -1, :]
+
+    def get_all_position_states(self, text: str) -> torch.Tensor:
+        """
+        Returns: Tensor of shape (seq_len, 960) for all positions
+        """
+        tokens = self.tokenizer(text, return_tensors="pt")
+        with torch.no_grad():
+            outputs = self.model(**tokens, output_hidden_states=True)
+        return outputs.hidden_states[self.layer][0]
+```
+
+---
+
+## 7. Complete Weight Inventory Table
+
+### 7.1 All Named Parameters
+
+```
+EMBEDDING (47,185,920 params - 13.04%)
+  model.embed_tokens.weight                    (49152, 960)    47,185,920
+
+LAYER 0 (9,832,320 params - 2.72%)
+  Attention (2,457,600):
+    model.layers.0.self_attn.q_proj.weight     (960, 960)      921,600
+    model.layers.0.self_attn.k_proj.weight     (320, 960)      307,200
+    model.layers.0.self_attn.v_proj.weight     (320, 960)      307,200
+    model.layers.0.self_attn.o_proj.weight     (960, 960)      921,600
+  MLP (7,372,800):
+    model.layers.0.mlp.gate_proj.weight        (2560, 960)     2,457,600
+    model.layers.0.mlp.up_proj.weight          (2560, 960)     2,457,600
+    model.layers.0.mlp.down_proj.weight        (960, 2560)     2,457,600
+  Norms (1,920):
+    model.layers.0.input_layernorm.weight      (960,)          960
+    model.layers.0.post_attention_layernorm.weight (960,)      960
+
+[Layers 1-31 follow identical structure, each with 9,832,320 params]
+
+FINAL NORM (960 params - 0.00%)
+  model.norm.weight                            (960,)          960
+
+LM HEAD (tied with embed_tokens)
+  lm_head.weight                               (49152, 960)    [shared]
+```
+
+### 7.2 Summary by Component Type
+
+| Component Type | Count | Params Each | Total Params |
+|----------------|-------|-------------|--------------|
+| Embedding | 1 | 47,185,920 | 47,185,920 |
+| Q Projection | 32 | 921,600 | 29,491,200 |
+| K Projection | 32 | 307,200 | 9,830,400 |
+| V Projection | 32 | 307,200 | 9,830,400 |
+| O Projection | 32 | 921,600 | 29,491,200 |
+| Gate Projection | 32 | 2,457,600 | 78,643,200 |
+| Up Projection | 32 | 2,457,600 | 78,643,200 |
+| Down Projection | 32 | 2,457,600 | 78,643,200 |
+| Input LayerNorm | 32 | 960 | 30,720 |
+| Post-Attn LayerNorm | 32 | 960 | 30,720 |
+| Final LayerNorm | 1 | 960 | 960 |
+| **Total** | | | **361,821,120** |
+
+---
+
+## 8. Configuration Reference
+
+Complete model configuration from HuggingFace:
+
+```python
+{
+    "architectures": ["LlamaForCausalLM"],
+    "attention_bias": False,
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "pad_token_id": 2,
+    "head_dim": 64,
+    "hidden_act": "silu",
+    "hidden_size": 960,
+    "initializer_range": 0.02,
+    "intermediate_size": 2560,
+    "max_position_embeddings": 8192,
+    "mlp_bias": False,
+    "model_type": "llama",
+    "num_attention_heads": 15,
+    "num_hidden_layers": 32,
+    "num_key_value_heads": 5,
+    "pretraining_tp": 1,
+    "rms_norm_eps": 1e-05,
+    "rope_interleaved": False,
+    "rope_theta": 100000,
+    "tie_word_embeddings": True,
+    "use_cache": True,
+    "vocab_size": 49152
+}
+```
+
+---
+
+## 9. Appendix: PyTorch Model Structure
+
+```
+LlamaForCausalLM(
+  (model): LlamaModel(
+    (embed_tokens): Embedding(49152, 960, padding_idx=2)
+    (layers): ModuleList(
+      (0-31): 32 x LlamaDecoderLayer(
+        (self_attn): LlamaAttention(
+          (q_proj): Linear(in_features=960, out_features=960, bias=False)
+          (k_proj): Linear(in_features=960, out_features=320, bias=False)
+          (v_proj): Linear(in_features=960, out_features=320, bias=False)
+          (o_proj): Linear(in_features=960, out_features=960, bias=False)
+        )
+        (mlp): LlamaMLP(
+          (gate_proj): Linear(in_features=960, out_features=2560, bias=False)
+          (up_proj): Linear(in_features=960, out_features=2560, bias=False)
+          (down_proj): Linear(in_features=2560, out_features=960, bias=False)
+          (act_fn): SiLUActivation()
+        )
+        (input_layernorm): LlamaRMSNorm((960,), eps=1e-05)
+        (post_attention_layernorm): LlamaRMSNorm((960,), eps=1e-05)
+      )
+    )
+    (norm): LlamaRMSNorm((960,), eps=1e-05)
+    (rotary_emb): LlamaRotaryEmbedding()
+  )
+  (lm_head): Linear(in_features=960, out_features=49152, bias=False)
+)
+```

llm_integration/smollm2/analyze_smollm2.py

@@ -0,0 +1,232 @@
+"""
+SmolLM2-360M-Instruct Architecture Analysis
+For 8bit-threshold-computer LLM Integration Project
+"""
+
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig
+from collections import defaultdict
+import json
+
+def analyze_smollm2():
+    model_name = "HuggingFaceTB/SmolLM2-360M-Instruct"
+
+    print("=" * 80)
+    print("SmolLM2-360M-Instruct Architecture Analysis")
+    print("=" * 80)
+
+    # Load config first
+    print("\n[1] Loading model configuration...")
+    config = AutoConfig.from_pretrained(model_name)
+    print(f"Config loaded: {type(config).__name__}")
+
+    # Load tokenizer
+    print("\n[2] Loading tokenizer...")
+    tokenizer = AutoTokenizer.from_pretrained(model_name)
+    print(f"Tokenizer loaded: {type(tokenizer).__name__}")
+
+    # Load model with hidden states output
+    print("\n[3] Loading model with output_hidden_states=True...")
+    model = AutoModelForCausalLM.from_pretrained(
+        model_name,
+        output_hidden_states=True,
+        torch_dtype=torch.float32
+    )
+    model.eval()
+    print(f"Model loaded: {type(model).__name__}")
+
+    # ========================================================================
+    # ARCHITECTURE CENSUS
+    # ========================================================================
+    print("\n" + "=" * 80)
+    print("ARCHITECTURE CENSUS")
+    print("=" * 80)
+
+    print("\n--- Model Configuration ---")
+    config_dict = config.to_dict()
+    for key, value in sorted(config_dict.items()):
+        print(f"  {key}: {value}")
+
+    print("\n--- Key Architecture Parameters ---")
+    print(f"  Model type: {config.model_type}")
+    print(f"  Vocabulary size: {config.vocab_size}")
+    print(f"  Hidden size: {config.hidden_size}")
+    print(f"  Intermediate size: {config.intermediate_size}")
+    print(f"  Number of hidden layers: {config.num_hidden_layers}")
+    print(f"  Number of attention heads: {config.num_attention_heads}")
+    print(f"  Number of KV heads: {getattr(config, 'num_key_value_heads', config.num_attention_heads)}")
+    print(f"  Head dimension: {config.hidden_size // config.num_attention_heads}")
+    print(f"  Max position embeddings: {config.max_position_embeddings}")
+    print(f"  RMS norm epsilon: {getattr(config, 'rms_norm_eps', 'N/A')}")
+    print(f"  Rope theta: {getattr(config, 'rope_theta', 'N/A')}")
+    print(f"  Tie word embeddings: {getattr(config, 'tie_word_embeddings', 'N/A')}")
+
+    # ========================================================================
+    # WEIGHT INVENTORY
+    # ========================================================================
+    print("\n" + "=" * 80)
+    print("WEIGHT INVENTORY")
+    print("=" * 80)
+
+    total_params = 0
+    param_groups = defaultdict(list)
+
+    for name, param in model.named_parameters():
+        total_params += param.numel()
+
+        # Group by component
+        if "embed_tokens" in name:
+            group = "Embedding"
+        elif "lm_head" in name:
+            group = "LM Head"
+        elif "norm" in name and "layers" not in name:
+            group = "Final Norm"
+        elif "layers" in name:
+            layer_num = name.split(".")[2]
+            if "self_attn" in name:
+                group = f"Layer {layer_num} - Attention"
+            elif "mlp" in name:
+                group = f"Layer {layer_num} - MLP"
+            elif "norm" in name:
+                group = f"Layer {layer_num} - Norms"
+            else:
+                group = f"Layer {layer_num} - Other"
+        else:
+            group = "Other"
+
+        param_groups[group].append({
+            "name": name,
+            "shape": tuple(param.shape),
+            "numel": param.numel(),
+            "dtype": str(param.dtype)
+        })
+
+    print(f"\n--- Total Parameters: {total_params:,} ---")
+    print(f"    ({total_params / 1e6:.2f}M parameters)")
+
+    # Print by group
+    for group_name in sorted(param_groups.keys()):
+        params = param_groups[group_name]
+        group_total = sum(p["numel"] for p in params)
+        print(f"\n### {group_name} ({group_total:,} params, {group_total/total_params*100:.2f}%)")
+        for p in params:
+            print(f"    {p['name']}")
+            print(f"        Shape: {p['shape']}, Elements: {p['numel']:,}, Dtype: {p['dtype']}")
+
+    # ========================================================================
+    # TOKENIZATION ANALYSIS
+    # ========================================================================
+    print("\n" + "=" * 80)
+    print("TOKENIZATION ANALYSIS")
+    print("=" * 80)
+
+    test_input = "47 + 86"
+    print(f"\n--- Test Input: '{test_input}' ---")
+
+    tokens = tokenizer(test_input, return_tensors="pt")
+    input_ids = tokens["input_ids"][0]
+
+    print(f"\nInput IDs: {input_ids.tolist()}")
+    print(f"Number of tokens: {len(input_ids)}")
+
+    print("\nToken breakdown:")
+    for i, token_id in enumerate(input_ids):
+        token_str = tokenizer.decode([token_id])
+        print(f"  Position {i}: ID={token_id.item():5d}, Token='{token_str}'")
+
+    # Additional tokenization tests
+    print("\n--- Additional Tokenization Tests ---")
+    test_cases = ["0", "1", "47", "86", "133", " + ", "="]
+    for tc in test_cases:
+        ids = tokenizer.encode(tc, add_special_tokens=False)
+        decoded = [tokenizer.decode([i]) for i in ids]
+        print(f"  '{tc}' -> IDs: {ids}, Tokens: {decoded}")
+
+    # ========================================================================
+    # HIDDEN STATE ANALYSIS
+    # ========================================================================
+    print("\n" + "=" * 80)
+    print("HIDDEN STATE ANALYSIS")
+    print("=" * 80)
+
+    print(f"\n--- Running inference on '{test_input}' ---")
+
+    with torch.no_grad():
+        outputs = model(**tokens)
+
+    hidden_states = outputs.hidden_states
+    print(f"\nNumber of hidden state outputs: {len(hidden_states)}")
+    print("(This includes embedding output + each layer's output)")
+
+    print("\nHidden state shapes at each layer:")
+    for i, hs in enumerate(hidden_states):
+        layer_name = "Embedding" if i == 0 else f"Layer {i-1}"
+        print(f"  {layer_name}: {tuple(hs.shape)}")
+        if i == 0:
+            print(f"      (batch_size=1, seq_len={hs.shape[1]}, hidden_dim={hs.shape[2]})")
+
+    # Analyze hidden state statistics at different layers
+    print("\n--- Hidden State Statistics (per layer) ---")
+    for i, hs in enumerate(hidden_states):
+        layer_name = "Embedding" if i == 0 else f"Layer {i-1}"
+        hs_flat = hs.view(-1)
+        print(f"  {layer_name}:")
+        print(f"      Mean: {hs_flat.mean().item():.6f}")
+        print(f"      Std:  {hs_flat.std().item():.6f}")
+        print(f"      Min:  {hs_flat.min().item():.6f}")
+        print(f"      Max:  {hs_flat.max().item():.6f}")
+
+    # ========================================================================
+    # MODEL STRUCTURE DEEP DIVE
+    # ========================================================================
+    print("\n" + "=" * 80)
+    print("MODEL STRUCTURE DEEP DIVE")
+    print("=" * 80)
+
+    print("\n--- Model Architecture String ---")
+    print(model)
+
+    # ========================================================================
+    # SUMMARY DATA FOR REPORT
+    # ========================================================================
+    summary = {
+        "model_name": model_name,
+        "total_params": total_params,
+        "config": {
+            "vocab_size": config.vocab_size,
+            "hidden_size": config.hidden_size,
+            "intermediate_size": config.intermediate_size,
+            "num_hidden_layers": config.num_hidden_layers,
+            "num_attention_heads": config.num_attention_heads,
+            "num_kv_heads": getattr(config, 'num_key_value_heads', config.num_attention_heads),
+            "head_dim": config.hidden_size // config.num_attention_heads,
+            "max_position_embeddings": config.max_position_embeddings,
+            "rms_norm_eps": getattr(config, 'rms_norm_eps', None),
+            "rope_theta": getattr(config, 'rope_theta', None),
+            "tie_word_embeddings": getattr(config, 'tie_word_embeddings', None),
+        },
+        "tokenization": {
+            "test_input": test_input,
+            "token_ids": input_ids.tolist(),
+            "num_tokens": len(input_ids),
+            "tokens": [tokenizer.decode([tid]) for tid in input_ids]
+        },
+        "hidden_states": {
+            "num_outputs": len(hidden_states),
+            "shape": list(hidden_states[0].shape)
+        },
+        "param_groups": {k: {"count": len(v), "total": sum(p["numel"] for p in v)} for k, v in param_groups.items()}
+    }
+
+    # Save summary as JSON for report generation
+    with open("D:/8bit-threshold-computer/llm_integration/smollm2_analysis.json", "w") as f:
+        json.dump(summary, f, indent=2)
+
+    print("\n" + "=" * 80)
+    print("Analysis complete. Summary saved to smollm2_analysis.json")
+    print("=" * 80)
+
+    return summary, model, tokenizer, hidden_states, param_groups
+
+if __name__ == "__main__":
+    summary, model, tokenizer, hidden_states, param_groups = analyze_smollm2()

llm_integration/smollm2/smollm2_analysis.json

@@ -0,0 +1,439 @@
+{
+  "model_name": "HuggingFaceTB/SmolLM2-360M-Instruct",
+  "total_params": 361821120,
+  "config": {
+    "vocab_size": 49152,
+    "hidden_size": 960,
+    "intermediate_size": 2560,
+    "num_hidden_layers": 32,
+    "num_attention_heads": 15,
+    "num_kv_heads": 5,
+    "head_dim": 64,
+    "max_position_embeddings": 8192,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 100000,
+    "tie_word_embeddings": true
+  },
+  "tokenization": {
+    "test_input": "47 + 86",
+    "token_ids": [
+      36,
+      39,
+      1232,
+      216,
+      40,
+      38
+    ],
+    "num_tokens": 6,
+    "tokens": [
+      "4",
+      "7",
+      " +",
+      " ",
+      "8",
+      "6"
+    ]
+  },
+  "hidden_states": {
+    "num_outputs": 33,
+    "shape": [
+      1,
+      6,
+      960
+    ]
+  },
+  "param_groups": {
+    "Embedding": {
+      "count": 1,
+      "total": 47185920
+    },
+    "Layer 0 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 0 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 0 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 1 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 1 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 1 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 2 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 2 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 2 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 3 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 3 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 3 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 4 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 4 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 4 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 5 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 5 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 5 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 6 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 6 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 6 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 7 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 7 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 7 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 8 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 8 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 8 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 9 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 9 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 9 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 10 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 10 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 10 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 11 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 11 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 11 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 12 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 12 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 12 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 13 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 13 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 13 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 14 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 14 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 14 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 15 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 15 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 15 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 16 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 16 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 16 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 17 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 17 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 17 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 18 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 18 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 18 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 19 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 19 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 19 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 20 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 20 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 20 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 21 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 21 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 21 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 22 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 22 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 22 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 23 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 23 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 23 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 24 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 24 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 24 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 25 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 25 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 25 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 26 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 26 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 26 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 27 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 27 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 27 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 28 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 28 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 28 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 29 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 29 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 29 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 30 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 30 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 30 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Layer 31 - Attention": {
+      "count": 4,
+      "total": 2457600
+    },
+    "Layer 31 - MLP": {
+      "count": 3,
+      "total": 7372800
+    },
+    "Layer 31 - Norms": {
+      "count": 2,
+      "total": 1920
+    },
+    "Final Norm": {
+      "count": 1,
+      "total": 960
+    }
+  }
+}

llm_integration/train.py

@@ -0,0 +1,955 @@
+"""
+Unified training script for threshold circuit LLM integration.
+
+Modes:
+  --mode router    : Train only OpRouter with ground truth bits (sanity check)
+  --mode interface : Train BitEncoder + OpRouter with ground truth bits (sanity check)
+  --mode llm       : Train extractor with LLM hidden states (the real training)
+
+LLM mode options:
+  --unfreeze_layers N : Unfreeze top N transformer layers (default 0 = fully frozen)
+
+Hardware Profile (NVIDIA RTX 6000 Ada 48GB):
+  VRAM Scaling (unfreeze_layers=4):
+    batch_size |  VRAM   |  %
+    -----------+---------+------
+         512   |  5,784  | 11.8%
+       1,024   |  7,384  | 15.0%
+       4,096   | 16,534  | 33.6%
+      13,000   | 39,000  | 79.4%  <-- recommended for 80% target
+
+Examples:
+  python train.py --mode llm --epochs 100 --batch_size 256
+  python train.py --mode llm --epochs 100 --batch_size 4096 --unfreeze_layers 4
+"""
+
+import torch
+import torch.nn as nn
+import torch.optim as optim
+import time
+import argparse
+import random
+
+from model import (
+    ThresholdALU, DirectCircuitModel, OpRouter,
+    ArithmeticModel, OPERATIONS, OP_SYMBOLS
+)
+from circuits import FrozenThresholdCircuits
+from fitness import generate_batch, compute_fitness, compute_loss
+
+DEVICE = 'cuda'
+
+ONES = ['zero', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight', 'nine',
+        'ten', 'eleven', 'twelve', 'thirteen', 'fourteen', 'fifteen', 'sixteen',
+        'seventeen', 'eighteen', 'nineteen']
+TENS = ['', '', 'twenty', 'thirty', 'forty', 'fifty', 'sixty', 'seventy', 'eighty', 'ninety']
+
+def int_to_words(n: int) -> str:
+    """Convert integer 0-255 to English words."""
+    if n < 0 or n > 255:
+        return str(n)
+    if n < 20:
+        return ONES[n]
+    if n < 100:
+        if n % 10 == 0:
+            return TENS[n // 10]
+        return f"{TENS[n // 10]} {ONES[n % 10]}"
+    if n % 100 == 0:
+        return f"{ONES[n // 100]} hundred"
+    if n % 100 < 20:
+        return f"{ONES[n // 100]} hundred {ONES[n % 100]}"
+    if n % 10 == 0:
+        return f"{ONES[n // 100]} hundred {TENS[(n % 100) // 10]}"
+    return f"{ONES[n // 100]} hundred {TENS[(n % 100) // 10]} {ONES[n % 10]}"
+
+
+def int_to_bits(val: int, device: str = 'cuda') -> torch.Tensor:
+    bits = torch.zeros(8, device=device)
+    for i in range(8):
+        bits[7-i] = (val >> i) & 1
+    return bits
+
+
+def bits_to_int(bits: torch.Tensor) -> int:
+    val = 0
+    for i in range(8):
+        if bits[i].item() > 0.5:
+            val += 1 << (7-i)
+    return val
+
+
+NL_TEMPLATES = {
+    'add': [
+        "What is {a} plus {b}?",
+        "Calculate {a} + {b}",
+        "Add {a} and {b}",
+        "What's the sum of {a} and {b}?",
+        "If I have {a} and get {b} more, how many total?",
+        "{a} + {b} = ?",
+        "Compute {a} plus {b}",
+    ],
+    'sub': [
+        "What is {a} minus {b}?",
+        "Calculate {a} - {b}",
+        "Subtract {b} from {a}",
+        "What's {a} take away {b}?",
+        "If I have {a} and lose {b}, how many left?",
+        "{a} - {b} = ?",
+        "Compute {a} minus {b}",
+    ],
+    'mul': [
+        "What is {a} times {b}?",
+        "Calculate {a} * {b}",
+        "Multiply {a} by {b}",
+        "What's {a} multiplied by {b}?",
+        "{a} * {b} = ?",
+        "Compute {a} times {b}",
+        "What is the product of {a} and {b}?",
+    ],
+    'gt': [
+        "Is {a} greater than {b}?",
+        "Is {a} > {b}?",
+        "Check if {a} is larger than {b}",
+        "Compare: is {a} more than {b}?",
+        "{a} > {b}?",
+    ],
+    'lt': [
+        "Is {a} less than {b}?",
+        "Is {a} < {b}?",
+        "Check if {a} is smaller than {b}",
+        "Compare: is {a} fewer than {b}?",
+        "{a} < {b}?",
+    ],
+    'eq': [
+        "Is {a} equal to {b}?",
+        "Is {a} == {b}?",
+        "Does {a} equal {b}?",
+        "Check if {a} equals {b}",
+        "Are {a} and {b} the same?",
+    ],
+}
+
+
+def generate_problem(max_val: int = 255):
+    """
+    Generate a random arithmetic problem for LLM training.
+    Randomly mixes digit and word formats.
+    """
+    a = random.randint(0, max_val)
+    b = random.randint(0, max_val)
+    op = random.choice(OPERATIONS)
+
+    fmt = random.choice(['digits', 'words', 'nl_digits', 'nl_words'])
+
+    if fmt == 'digits':
+        sym = OP_SYMBOLS[op]
+        text = f"{a} {sym} {b}"
+    elif fmt == 'words':
+        a_word = int_to_words(a)
+        b_word = int_to_words(b)
+        op_word = {'add': 'plus', 'sub': 'minus', 'mul': 'times',
+                   'gt': 'greater than', 'lt': 'less than', 'eq': 'equals'}[op]
+        text = f"{a_word} {op_word} {b_word}"
+    elif fmt == 'nl_digits':
+        template = random.choice(NL_TEMPLATES[op])
+        text = template.format(a=a, b=b)
+    else:
+        template = random.choice(NL_TEMPLATES[op])
+        text = template.format(a=int_to_words(a), b=int_to_words(b))
+
+    if op == 'add':
+        result = (a + b) & 0xFF
+    elif op == 'sub':
+        result = (a - b) & 0xFF
+    elif op == 'mul':
+        result = (a * b) & 0xFF
+    elif op == 'gt':
+        result = 1 if a > b else 0
+    elif op == 'lt':
+        result = 1 if a < b else 0
+    elif op == 'eq':
+        result = 1 if a == b else 0
+
+    return text, a, b, op, result
+
+
+def get_curriculum_max(epoch: int, total_epochs: int) -> int:
+    """
+    Curriculum learning: start with small numbers, gradually increase.
+    Epoch 0-20%: 0-9 (single digit)
+    Epoch 20-50%: 0-99 (two digit)
+    Epoch 50-100%: 0-255 (full range)
+    """
+    progress = epoch / total_epochs
+    if progress < 0.2:
+        return 9
+    elif progress < 0.5:
+        return 99
+    else:
+        return 255
+
+
+def train_router(epochs: int = 100, batch_size: int = 256, lr: float = 1e-2, device: str = 'cuda'):
+    """Train only the router with ground truth bits."""
+    print("=" * 70)
+    print(" ROUTER-ONLY TRAINING (Ground Truth Bits)")
+    print("=" * 70)
+
+    circuits = FrozenThresholdCircuits(device=device)
+    router = OpRouter(input_dim=16 + 6, hidden_dim=64, n_ops=6).to(device)
+
+    print(f"\nRouter parameters: {sum(p.numel() for p in router.parameters()):,}")
+
+    def model_fn(a_bits, b_bits, op_onehot):
+        x = torch.cat([a_bits, b_bits, op_onehot], dim=-1)
+        op_weights = router(x)
+        return circuits(a_bits, b_bits, op_weights)
+
+    initial_fitness = compute_fitness(model_fn, n_samples=1000, device=device)
+    print(f"Initial fitness: {initial_fitness:.4f}")
+
+    optimizer = optim.AdamW(router.parameters(), lr=lr)
+    scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=epochs)
+
+    print("\nTraining...")
+    print("-" * 70)
+
+    best_fitness = initial_fitness
+    start_time = time.perf_counter()
+
+    for epoch in range(epochs):
+        router.train()
+        epoch_loss = 0.0
+
+        for _ in range(100):
+            batch = generate_batch(batch_size, device)
+
+            optimizer.zero_grad()
+
+            x = torch.cat([batch['a_bits'], batch['b_bits'], batch['op_onehot']], dim=-1)
+            op_weights = router(x)
+            pred_bits = circuits(batch['a_bits'], batch['b_bits'], op_weights)
+
+            loss = compute_loss(pred_bits, batch['expected_bits'])
+            loss.backward()
+            optimizer.step()
+
+            epoch_loss += loss.item()
+
+        scheduler.step()
+
+        if (epoch + 1) % 10 == 0 or epoch == 0:
+            router.eval()
+            fitness, details = compute_fitness(model_fn, n_samples=2000, device=device, return_details=True)
+            elapsed = time.perf_counter() - start_time
+
+            if fitness > best_fitness:
+                best_fitness = fitness
+                marker = " *"
+            else:
+                marker = ""
+
+            print(f"Epoch {epoch+1:3d} | Loss: {epoch_loss/100:.4f} | "
+                  f"Fitness: {fitness:.4f}{marker} | Time: {elapsed:.1f}s")
+
+            if fitness >= 0.9999:
+                print("\n TARGET: 100% FITNESS ACHIEVED")
+                break
+
+    print("\n" + "=" * 70)
+    print(" RESULTS")
+    print("=" * 70)
+
+    router.eval()
+    final_fitness, details = compute_fitness(model_fn, n_samples=5000, device=device, return_details=True)
+
+    print(f"\nFinal fitness: {final_fitness:.4f}")
+    print(f"\nPer-operation:")
+    for op in OPERATIONS:
+        acc = details['by_op'][op]['accuracy']
+        print(f"  {op}: {acc:.4f}")
+
+    print(f"\nTotal time: {time.perf_counter() - start_time:.1f}s")
+
+    if final_fitness >= 0.99:
+        print("\nCONCLUSION: Router successfully learned operation dispatch.")
+        print("           With correct bit encoding, 100% is achievable.")
+
+    save_path = "D:/8bit-threshold-computer/llm_integration/trained/router.pt"
+    torch.save({
+        'router_state_dict': router.state_dict(),
+        'final_fitness': final_fitness,
+        'params': sum(p.numel() for p in router.parameters()),
+    }, save_path)
+    print(f"\nSaved trained router to: {save_path}")
+
+    return router, final_fitness
+
+
+def get_gpu_memory():
+    """Get GPU memory usage in MB."""
+    if torch.cuda.is_available():
+        return torch.cuda.memory_allocated() / 1024 / 1024, torch.cuda.max_memory_allocated() / 1024 / 1024
+    return 0, 0
+
+
+def train_interface(epochs: int = 200, batch_size: int = 512, lr: float = 1e-3,
+                    eval_interval: int = 10, device: str = 'cuda'):
+    """Train BitEncoder + OpRouter with ground truth bits."""
+    print("=" * 70)
+    print(" INTERFACE TRAINING (Encoder + Router)")
+    print("=" * 70)
+    print(f"  Started at: {time.strftime('%H:%M:%S')}")
+
+    print("\n[1/4] Verifying frozen circuits...")
+    print("  Creating DirectCircuitModel...", end=" ", flush=True)
+    direct_model = DirectCircuitModel(device=device)
+    mem, max_mem = get_gpu_memory()
+    print(f"done. VRAM: {mem:.0f}MB")
+
+    def direct_fn(a, b, op):
+        return direct_model(a, b, op)
+
+    print("  Running fitness check (1000 samples)...", end=" ", flush=True)
+    circuit_fitness = compute_fitness(direct_fn, n_samples=1000, device=device)
+    print(f"done. Fitness: {circuit_fitness:.4f}")
+    if circuit_fitness < 0.999:
+        print("  ERROR: Circuits not achieving 100%. Aborting.")
+        return None, 0.0
+    print("  STATUS: PASS")
+
+    print("\n[2/4] Initializing model...")
+    print("  Creating ThresholdALU...", end=" ", flush=True)
+    model = ThresholdALU(device=device)
+    mem, max_mem = get_gpu_memory()
+    print(f"done. VRAM: {mem:.0f}MB")
+
+    trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
+    print(f"  Trainable parameters: {trainable_params:,}")
+
+    def model_fn(a, b, op):
+        return model(a, b, op)
+
+    print("  Running initial fitness check...", end=" ", flush=True)
+    initial_fitness = compute_fitness(model_fn, n_samples=1000, device=device)
+    print(f"done. Fitness: {initial_fitness:.4f}")
+
+    print("\n[3/4] Setting up training...")
+    print("  Creating optimizer...", end=" ", flush=True)
+    optimizer = optim.AdamW(model.parameters(), lr=lr)
+    print("done.")
+    print("  Creating scheduler...", end=" ", flush=True)
+    scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=epochs)
+    print("done.")
+
+    print(f"  Config: lr={lr}, batch_size={batch_size}, epochs={epochs}")
+
+    print("\n[4/4] Training...")
+    print("  Generating first batch to warm up...", end=" ", flush=True)
+    warmup_batch = generate_batch(batch_size, device)
+    mem, max_mem = get_gpu_memory()
+    print(f"done. VRAM: {mem:.0f}MB (max: {max_mem:.0f}MB)")
+
+    print("-" * 70)
+
+    best_fitness = initial_fitness
+    start_time = time.perf_counter()
+    n_batches = 100
+
+    for epoch in range(epochs):
+        model.train()
+        epoch_loss = 0.0
+        epoch_start = time.perf_counter()
+
+        for batch_idx in range(n_batches):
+            batch = generate_batch(batch_size, device)
+
+            optimizer.zero_grad()
+
+            pred_bits = model(batch['a_bits'], batch['b_bits'], batch['op_onehot'])
+
+            loss = compute_loss(pred_bits, batch['expected_bits'])
+
+            loss.backward()
+            optimizer.step()
+
+            epoch_loss += loss.item()
+
+            if batch_idx == 0 and epoch == 0:
+                mem, max_mem = get_gpu_memory()
+                print(f"  First forward/backward done. VRAM: {mem:.0f}MB (max: {max_mem:.0f}MB)")
+
+            if (batch_idx + 1) % 25 == 0:
+                avg_so_far = epoch_loss / (batch_idx + 1)
+                print(f"  Epoch {epoch+1} batch {batch_idx+1}/{n_batches} | loss: {avg_so_far:.4f}", flush=True)
+
+        scheduler.step()
+
+        avg_loss = epoch_loss / n_batches
+        epoch_time = time.perf_counter() - epoch_start
+
+        if (epoch + 1) % 5 == 0 or epoch == 0:  # Eval every 5 epochs
+            model.eval()
+            fitness, details = compute_fitness(
+                model_fn, n_samples=2000, device=device, return_details=True
+            )
+
+            elapsed = time.perf_counter() - start_time
+
+            if fitness > best_fitness:
+                best_fitness = fitness
+                marker = " *"
+            else:
+                marker = ""
+
+            mem, _ = get_gpu_memory()
+            print(f"Epoch {epoch+1:4d} | Loss: {avg_loss:.4f} | "
+                  f"Fitness: {fitness:.4f}{marker} | "
+                  f"LR: {scheduler.get_last_lr()[0]:.2e} | "
+                  f"VRAM: {mem:.0f}MB | "
+                  f"Time: {elapsed:.1f}s ({epoch_time:.1f}s/epoch)")
+
+            if fitness >= 0.9999:
+                print("\n" + "=" * 70)
+                print(" TARGET ACHIEVED: 100% FITNESS")
+                print("=" * 70)
+                break
+
+    print("\n" + "=" * 70)
+    print(" TRAINING COMPLETE")
+    print("=" * 70)
+
+    model.eval()
+    final_fitness, details = compute_fitness(
+        model_fn, n_samples=5000, device=device, return_details=True
+    )
+
+    print(f"\nFinal fitness: {final_fitness:.4f}")
+    print(f"Best fitness:  {best_fitness:.4f}")
+    print(f"\nPer-operation breakdown:")
+    for op in OPERATIONS:
+        acc = details['by_op'][op]['accuracy']
+        print(f"  {op:6}: {acc:.4f}")
+
+    print(f"\nTotal time: {time.perf_counter() - start_time:.1f}s")
+
+    save_path = "D:/8bit-threshold-computer/llm_integration/trained/interface.pt"
+    torch.save({
+        'encoder_state_dict': model.encoder.state_dict(),
+        'router_state_dict': model.router.state_dict(),
+        'final_fitness': final_fitness,
+        'best_fitness': best_fitness,
+    }, save_path)
+    print(f"\nSaved trained model to: {save_path}")
+
+    return model, final_fitness
+
+
+def compute_llm_loss(pred_bits, a_bits, b_bits, op_logits,
+                     target_result, target_a, target_b, target_op_idx,
+                     bit_weight: float = 2.0):
+    """
+    Multi-component loss for LLM training.
+    bit_weight: multiplier for a/b bit losses (default 2x since extraction is the bottleneck)
+    """
+    result_loss = nn.functional.binary_cross_entropy_with_logits(
+        pred_bits, target_result
+    )
+
+    a_bits_safe = torch.clamp(a_bits, 0.0, 1.0)
+    b_bits_safe = torch.clamp(b_bits, 0.0, 1.0)
+    a_bits_safe = torch.nan_to_num(a_bits_safe, nan=0.5, posinf=1.0, neginf=0.0)
+    b_bits_safe = torch.nan_to_num(b_bits_safe, nan=0.5, posinf=1.0, neginf=0.0)
+
+    a_loss = nn.functional.binary_cross_entropy(
+        torch.clamp(a_bits_safe, 1e-6, 1-1e-6), target_a
+    )
+    b_loss = nn.functional.binary_cross_entropy(
+        torch.clamp(b_bits_safe, 1e-6, 1-1e-6), target_b
+    )
+
+    op_loss = nn.functional.cross_entropy(op_logits, target_op_idx)
+
+    total = result_loss + bit_weight * a_loss + bit_weight * b_loss + op_loss
+    total = torch.nan_to_num(total, nan=10.0, posinf=10.0, neginf=0.0)
+
+    return total, {
+        'result': result_loss.item() if not torch.isnan(result_loss) else 10.0,
+        'a': a_loss.item() if not torch.isnan(a_loss) else 10.0,
+        'b': b_loss.item() if not torch.isnan(b_loss) else 10.0,
+        'op': op_loss.item() if not torch.isnan(op_loss) else 10.0
+    }
+
+
+def value_to_digits(value: int) -> list:
+    """Convert integer value to list of digits (hundreds, tens, ones)."""
+    digits = []
+    for place in [100, 10, 1]:
+        digit = (value // place) % 10
+        digits.append(digit)
+    return digits
+
+
+def compute_positional_digit_loss(pred_bits, op_logits, a_digit_logits_list, b_digit_logits_list,
+                                   target_result, target_op_idx, target_a_values, target_b_values,
+                                   device, digit_weight: float = 5.0):
+    """
+    Loss for positional digit extraction with DIRECT digit supervision.
+
+    This provides strong gradients by directly supervising digit classification
+    instead of going through the value -> bits conversion.
+    """
+    result_loss = nn.functional.binary_cross_entropy_with_logits(
+        pred_bits, target_result
+    )
+
+    op_loss = nn.functional.cross_entropy(op_logits, target_op_idx)
+
+    a_digit_loss = torch.tensor(0.0, device=device)
+    b_digit_loss = torch.tensor(0.0, device=device)
+    n_a_digits = 0
+    n_b_digits = 0
+
+    for i, (a_logits_list, b_logits_list) in enumerate(zip(a_digit_logits_list, b_digit_logits_list)):
+        target_a = target_a_values[i].item()
+        target_b = target_b_values[i].item()
+
+        a_digits = value_to_digits(int(target_a))
+        b_digits = value_to_digits(int(target_b))
+
+        n_a = len(a_logits_list)
+        n_b = len(b_logits_list)
+
+        if n_a > 0:
+            target_a_digits = a_digits[-n_a:]
+            for j, logits in enumerate(a_logits_list):
+                target_digit = torch.tensor([target_a_digits[j]], device=device, dtype=torch.long)
+                a_digit_loss = a_digit_loss + nn.functional.cross_entropy(logits.unsqueeze(0), target_digit)
+                n_a_digits += 1
+
+        if n_b > 0:
+            target_b_digits = b_digits[-n_b:]
+            for j, logits in enumerate(b_logits_list):
+                target_digit = torch.tensor([target_b_digits[j]], device=device, dtype=torch.long)
+                b_digit_loss = b_digit_loss + nn.functional.cross_entropy(logits.unsqueeze(0), target_digit)
+                n_b_digits += 1
+
+    if n_a_digits > 0:
+        a_digit_loss = a_digit_loss / n_a_digits
+    if n_b_digits > 0:
+        b_digit_loss = b_digit_loss / n_b_digits
+
+    total = result_loss + digit_weight * a_digit_loss + digit_weight * b_digit_loss + op_loss
+    total = torch.nan_to_num(total, nan=10.0, posinf=10.0, neginf=0.0)
+
+    return total, {
+        'result': result_loss.item() if not torch.isnan(result_loss) else 10.0,
+        'a_digit': a_digit_loss.item() if not torch.isnan(a_digit_loss) else 10.0,
+        'b_digit': b_digit_loss.item() if not torch.isnan(b_digit_loss) else 10.0,
+        'op': op_loss.item() if not torch.isnan(op_loss) else 10.0
+    }
+
+
+def compute_hybrid_loss(pred_bits, op_logits, used_lookup,
+                        a_digit_logits, b_digit_logits,
+                        target_result, target_a_values, target_b_values, target_op_idx,
+                        device, digit_weight: float = 2.0):
+    """
+    Loss for hybrid extraction with digit-level prediction.
+
+    Uses cross-entropy on each digit (hundreds, tens, ones) for word samples.
+    Samples using digit lookup are already 100% accurate - no loss computed.
+    """
+    result_loss = nn.functional.binary_cross_entropy_with_logits(
+        pred_bits, target_result
+    )
+
+    op_loss = nn.functional.cross_entropy(op_logits, target_op_idx)
+
+    word_mask = ~used_lookup
+    n_words = word_mask.sum().item()
+
+    if n_words > 0 and a_digit_logits is not None and b_digit_logits is not None:
+        target_a_word = target_a_values[word_mask].long()
+        target_b_word = target_b_values[word_mask].long()
+
+        a_hundreds = target_a_word // 100
+        a_tens = (target_a_word % 100) // 10
+        a_ones = target_a_word % 10
+
+        b_hundreds = target_b_word // 100
+        b_tens = (target_b_word % 100) // 10
+        b_ones = target_b_word % 10
+
+        a_logits = a_digit_logits.view(-1, 3, 10)
+        b_logits = b_digit_logits.view(-1, 3, 10)
+
+        a_digit_loss = (
+            nn.functional.cross_entropy(a_logits[:, 0], a_hundreds) +
+            nn.functional.cross_entropy(a_logits[:, 1], a_tens) +
+            nn.functional.cross_entropy(a_logits[:, 2], a_ones)
+        ) / 3.0
+
+        b_digit_loss = (
+            nn.functional.cross_entropy(b_logits[:, 0], b_hundreds) +
+            nn.functional.cross_entropy(b_logits[:, 1], b_tens) +
+            nn.functional.cross_entropy(b_logits[:, 2], b_ones)
+        ) / 3.0
+    else:
+        a_digit_loss = torch.tensor(0.0, device=device)
+        b_digit_loss = torch.tensor(0.0, device=device)
+
+    total = result_loss + op_loss + digit_weight * (a_digit_loss + b_digit_loss)
+    total = torch.nan_to_num(total, nan=10.0, posinf=10.0, neginf=0.0)
+
+    return total, {
+        'result': result_loss.item() if not torch.isnan(result_loss) else 10.0,
+        'a_digit': a_digit_loss.item() if not torch.isnan(a_digit_loss) else 10.0,
+        'b_digit': b_digit_loss.item() if not torch.isnan(b_digit_loss) else 10.0,
+        'op': op_loss.item() if not torch.isnan(op_loss) else 10.0,
+        'n_words': n_words,
+        'n_lookup': used_lookup.sum().item()
+    }
+
+
+def evaluate_llm(model, n_samples: int = 500):
+    """Evaluate LLM model on random problems (mixed digit/word format)."""
+    model.extractor.eval()
+    correct = 0
+    op_correct = 0
+
+    for _ in range(n_samples):
+        text, a, b, op, expected = generate_problem()
+
+        with torch.no_grad():
+            outputs = model([text])
+            result_bits = outputs[0]
+            op_logits = outputs[3]
+
+        pred_result = bits_to_int(result_bits[0])
+        pred_op = OPERATIONS[op_logits[0].argmax().item()]
+
+        if pred_result == expected:
+            correct += 1
+        if pred_op == op:
+            op_correct += 1
+
+    model.extractor.train()
+    return correct / n_samples, op_correct / n_samples
+
+
+def train_llm(epochs: int = 100, batch_size: int = 256, lr: float = 3e-4,
+              unfreeze_layers: int = 0, extract_layer: int = -1,
+              position_extract: bool = False, digit_pred: bool = False,
+              positional_digit: bool = False, device: str = 'cuda'):
+    """
+    Train extractor with LLM hidden states.
+
+    Args:
+        unfreeze_layers: Number of top transformer layers to unfreeze (0 = fully frozen)
+        extract_layer: Which layer to extract from (-1 = last)
+        position_extract: Use position-specific extraction (legacy)
+        digit_pred: Predict digits instead of bits (legacy)
+        positional_digit: Use positional digit extraction (legacy, 100% on digits only)
+    """
+    hybrid = not (positional_digit or position_extract or digit_pred)
+
+    print("=" * 70)
+    print(" LLM TRAINING")
+    if unfreeze_layers > 0:
+        print(f" {unfreeze_layers} transformer layers unfrozen")
+    else:
+        print(" LLM frozen")
+    if extract_layer != -1:
+        print(f" Extracting from layer {extract_layer}")
+    if hybrid:
+        print(" HYBRID extraction (digit lookup + word learning)")
+    elif positional_digit:
+        print(" POSITIONAL DIGIT extraction (legacy, 100% on digits only)")
+    elif position_extract:
+        print(" Position-specific extraction (legacy)")
+    elif digit_pred:
+        print(" Digit-level prediction (legacy)")
+    print("=" * 70)
+
+    print("\nInitializing model...")
+    model = ArithmeticModel(
+        device=device,
+        unfreeze_layers=unfreeze_layers,
+        extract_layer=extract_layer,
+        position_extract=position_extract,
+        digit_pred=digit_pred,
+        positional_digit=positional_digit,
+        hybrid=hybrid
+    )
+
+    optimizer = optim.AdamW(model.trainable_parameters(), lr=lr)
+    scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=epochs)
+
+    print(f"\nTraining config:")
+    print(f"  Epochs: {epochs}")
+    print(f"  Batch size: {batch_size}")
+    print(f"  Learning rate: {lr}")
+    print(f"  Unfreeze layers: {unfreeze_layers}")
+    print(f"  Samples/epoch: {batch_size * 20}")
+
+    print(f"\nInitial evaluation (200 samples)...")
+    acc, op_acc = evaluate_llm(model, 200)
+    print(f"  Accuracy: {acc:.4f}, Op accuracy: {op_acc:.4f}")
+
+    print(f"\nStarting training...")
+    print("-" * 70)
+
+    best_acc = acc
+    start_time = time.perf_counter()
+
+    for epoch in range(epochs):
+        model.extractor.train()
+        if unfreeze_layers > 0:
+            model.llm.train()
+
+        max_val = get_curriculum_max(epoch, epochs)
+
+        epoch_loss = 0
+        if hybrid:
+            epoch_losses = {'result': 0, 'a_digit': 0, 'b_digit': 0, 'op': 0, 'n_words': 0, 'n_lookup': 0}
+        elif positional_digit:
+            epoch_losses = {'result': 0, 'a_digit': 0, 'b_digit': 0, 'op': 0}
+        else:
+            epoch_losses = {'result': 0, 'a': 0, 'b': 0, 'op': 0}
+        n_batches = 20
+        epoch_start = time.perf_counter()
+
+        for batch_idx in range(n_batches):
+            batch_texts = []
+            batch_a = []
+            batch_b = []
+            batch_op = []
+            batch_result = []
+            batch_a_values = []
+            batch_b_values = []
+
+            for _ in range(batch_size):
+                text, a, b, op, result = generate_problem(max_val)
+                batch_texts.append(text)
+                batch_a.append(int_to_bits(a, device))
+                batch_b.append(int_to_bits(b, device))
+                batch_op.append(OPERATIONS.index(op))
+                batch_result.append(int_to_bits(result, device))
+                batch_a_values.append(a)
+                batch_b_values.append(b)
+
+            target_a = torch.stack(batch_a)
+            target_b = torch.stack(batch_b)
+            target_op = torch.tensor(batch_op, device=device)
+            target_result = torch.stack(batch_result)
+            target_a_values = torch.tensor(batch_a_values, device=device, dtype=torch.float32)
+            target_b_values = torch.tensor(batch_b_values, device=device, dtype=torch.float32)
+
+            optimizer.zero_grad()
+
+            outputs = model(batch_texts)
+            pred_bits, a_bits, b_bits, op_logits = outputs[0], outputs[1], outputs[2], outputs[3]
+
+            if hybrid:
+                a_values, b_values, used_lookup = outputs[4], outputs[5], outputs[6]
+                a_digit_logits, b_digit_logits = outputs[7], outputs[8]
+                loss, losses = compute_hybrid_loss(
+                    pred_bits, op_logits, used_lookup,
+                    a_digit_logits, b_digit_logits,
+                    target_result, target_a_values, target_b_values, target_op, device
+                )
+            elif positional_digit:
+                a_digit_logits_list = outputs[7]
+                b_digit_logits_list = outputs[8]
+                loss, losses = compute_positional_digit_loss(
+                    pred_bits, op_logits, a_digit_logits_list, b_digit_logits_list,
+                    target_result, target_op, target_a_values, target_b_values, device
+                )
+            else:
+                loss, losses = compute_llm_loss(
+                    pred_bits, a_bits, b_bits, op_logits,
+                    target_result, target_a, target_b, target_op
+                )
+
+            loss.backward()
+            torch.nn.utils.clip_grad_norm_(model.trainable_parameters(), 1.0)
+            optimizer.step()
+
+            epoch_loss += loss.item()
+            for k in epoch_losses:
+                epoch_losses[k] += losses[k]
+
+            if (batch_idx + 1) % 5 == 0:
+                avg_so_far = epoch_loss / (batch_idx + 1)
+                print(f"  Epoch {epoch+1} batch {batch_idx+1}/{n_batches} | loss: {avg_so_far:.4f}", flush=True)
+
+        epoch_time = time.perf_counter() - epoch_start
+        scheduler.step()
+
+        avg_loss = epoch_loss / n_batches
+        for k in epoch_losses:
+            epoch_losses[k] /= n_batches
+
+        acc, op_acc = evaluate_llm(model, 300)
+        elapsed = time.perf_counter() - start_time
+
+        marker = " *" if acc > best_acc else ""
+        if acc > best_acc:
+            best_acc = acc
+
+        mem, _ = get_gpu_memory()
+        print(f"Epoch {epoch+1:3d} | Loss: {avg_loss:.4f} | "
+              f"Acc: {acc:.4f}{marker} | OpAcc: {op_acc:.4f} | "
+              f"Range: 0-{max_val} | VRAM: {mem:.0f}MB | Time: {elapsed:.0f}s")
+        if hybrid:
+            print(f"          Losses - result:{epoch_losses['result']:.4f} "
+                  f"a_digit:{epoch_losses['a_digit']:.4f} b_digit:{epoch_losses['b_digit']:.4f} "
+                  f"op:{epoch_losses['op']:.4f} | words:{epoch_losses['n_words']:.0f} lookup:{epoch_losses['n_lookup']:.0f}")
+        elif positional_digit:
+            print(f"          Losses - result:{epoch_losses['result']:.4f} "
+                  f"a_digit:{epoch_losses['a_digit']:.4f} b_digit:{epoch_losses['b_digit']:.4f} "
+                  f"op:{epoch_losses['op']:.4f}")
+        else:
+            print(f"          Losses - result:{epoch_losses['result']:.4f} "
+                  f"a:{epoch_losses['a']:.4f} b:{epoch_losses['b']:.4f} "
+                  f"op:{epoch_losses['op']:.4f}")
+
+        if acc >= 0.99:
+            print("\n" + "=" * 70)
+            print(" TARGET ACHIEVED: 99%+ ACCURACY")
+            print("=" * 70)
+            break
+
+    print("\n" + "=" * 70)
+    print(" FINAL EVALUATION")
+    print("=" * 70)
+
+    acc, op_acc = evaluate_llm(model, 1000)
+    print(f"Final accuracy: {acc:.4f}")
+    print(f"Final op accuracy: {op_acc:.4f}")
+    print(f"Best accuracy: {best_acc:.4f}")
+
+    print("\nSample predictions:")
+    for _ in range(10):
+        text, a, b, op, expected = generate_problem()
+        with torch.no_grad():
+            outputs = model([text])
+            result_bits, a_bits, b_bits, op_logits = outputs[0], outputs[1], outputs[2], outputs[3]
+        pred = bits_to_int(result_bits[0])
+        pred_a = bits_to_int(a_bits[0])
+        pred_b = bits_to_int(b_bits[0])
+        pred_op = OPERATIONS[op_logits[0].argmax().item()]
+
+        status = "OK" if pred == expected else "WRONG"
+        print(f"  '{text}' = {expected} | pred={pred} (a={pred_a}, b={pred_b}, op={pred_op}) [{status}]")
+
+    save_path = "D:/8bit-threshold-computer/llm_integration/trained/llm.pt"
+    save_dict = {
+        'extractor_state_dict': model.extractor.state_dict(),
+        'final_accuracy': acc,
+        'best_accuracy': best_acc,
+        'unfreeze_layers': unfreeze_layers,
+    }
+    if unfreeze_layers > 0:
+        save_dict['llm_state_dict'] = {
+            k: v for k, v in model.llm.state_dict().items()
+            if any(f'layers.{i}.' in k for i in range(len(model.llm.model.layers) - unfreeze_layers, len(model.llm.model.layers)))
+        }
+    torch.save(save_dict, save_path)
+    print(f"\nSaved to: {save_path}")
+
+    return model, acc
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Unified training for threshold circuit LLM integration',
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Modes:
+  router    - Train only OpRouter with ground truth bits (sanity check)
+  interface - Train BitEncoder + OpRouter with ground truth bits (sanity check)
+  llm       - Train extractor with LLM hidden states (the real training)
+
+LLM options:
+  --unfreeze_layers N  Fine-tune top N transformer layers
+  --extract_layer N    Extract from layer N (-1 = last)
+  --position_extract   Use position-specific extraction
+  --digit_pred         Predict digits instead of bits
+
+Baked-in: curriculum learning (0-9 -> 0-99 -> 0-255), 2x loss weight for a/b
+
+Examples:
+  python train.py --mode llm --epochs 100
+  python train.py --mode llm --position_extract
+  python train.py --mode llm --digit_pred --extract_layer 12
+  python train.py --mode llm --unfreeze_layers 4 --batch_size 4096
+        """
+    )
+    parser.add_argument('--mode', type=str, required=True,
+                        choices=['router', 'interface', 'llm'],
+                        help='Training mode')
+    parser.add_argument('--epochs', type=int, default=100, help='Number of epochs')
+    parser.add_argument('--batch_size', type=int, default=512, help='Batch size (default: 512)')
+    parser.add_argument('--lr', type=float, default=None,
+                        help='Learning rate (default: mode-specific)')
+    parser.add_argument('--unfreeze_layers', type=int, default=0,
+                        help='Unfreeze top N transformer layers (default 0 = frozen)')
+    parser.add_argument('--extract_layer', type=int, default=0,
+                        help='Which layer to extract from (default: 0 = embeddings, best for digits)')
+    parser.add_argument('--position_extract', action='store_true',
+                        help='Use position-specific extraction (legacy)')
+    parser.add_argument('--digit_pred', action='store_true',
+                        help='Predict digits instead of bits (legacy)')
+    parser.add_argument('--positional_digit', action='store_true', default=False,
+                        help='Use positional digit extraction (legacy, 100%% on digits only)')
+    parser.add_argument('--device', type=str, default='cuda', help='Device')
+    args = parser.parse_args()
+
+    torch.manual_seed(42)
+    random.seed(42)
+
+    if args.mode == 'router':
+        lr = args.lr if args.lr is not None else 1e-2
+        train_router(epochs=args.epochs, batch_size=args.batch_size, lr=lr, device=args.device)
+
+    elif args.mode == 'interface':
+        lr = args.lr if args.lr is not None else 1e-3
+        model, fitness = train_interface(
+            epochs=args.epochs, batch_size=args.batch_size, lr=lr, device=args.device
+        )
+
+        print("\n" + "=" * 70)
+        print(" EXPERIMENT SUMMARY")
+        print("=" * 70)
+        print(f"\n  Control (Vanilla SmolLM2-360M):     11.90%")
+        print(f"  Experimental (Trained Interface):   {100*fitness:.2f}%")
+        if fitness > 0:
+            print(f"\n  Improvement: {100*(fitness - 0.119)/0.119:.1f}%")
+
+        if fitness >= 0.99:
+            print("\n  CONCLUSION: Frozen threshold circuits + trained interface")
+            print("              achieves near-perfect arithmetic accuracy.")
+            print("              Core thesis VALIDATED.")
+        else:
+            print(f"\n  CONCLUSION: Further training or architecture changes needed.")
+            print(f"              Current gap: {100*(1.0 - fitness):.2f}%")
+
+    elif args.mode == 'llm':
+        lr = args.lr if args.lr is not None else 3e-4
+        train_llm(
+            epochs=args.epochs,
+            batch_size=args.batch_size,
+            lr=lr,
+            unfreeze_layers=args.unfreeze_layers,
+            extract_layer=args.extract_layer,
+            position_extract=args.position_extract,
+            digit_pred=args.digit_pred,
+            positional_digit=args.positional_digit,
+            device=args.device
+        )
+
+
+if __name__ == "__main__":
+    main()

llm_integration/trained/llm.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69ff9e584d08c9707e2932bc0f62cf89cd7167a453065146252c6b7cdb57cbc4
+size 17296376

llm_integration/trained/router.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ddfc24cd4a98b65de8d434bb843ebd24f8c902d067201fd7954e7b623a8ebcd
+size 9811

machines.py

@@ -0,0 +1,1719 @@
+"""neural_subleq8 and neural_rv32 — runtimes, reference emulators, assemblers,
+an RV32 object loader, and test suites for the two endpoint machines of the
+family. One file; run the suites with:
+
+    python machines.py            # both machines
+    python machines.py subleq     # SUBLEQ-8 only
+    python machines.py rv32       # RV32 assembled-program suite
+    python machines.py rv32-c     # RV32 running stock-compiler C
+
+Build the model files with `python build.py --apply subleq` and
+`python build.py --apply rv32`.
+"""
+
+from __future__ import annotations
+import argparse
+import os
+import shutil
+import struct
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple
+
+import torch
+from safetensors import safe_open
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from eval import NetlistEvaluator, heaviside, load_metadata
+from eval_all import GenericThresholdALU, int_to_bits, bits_to_int
+
+# ============================================================================
+# RV32 relocatable-object loader
+# ============================================================================
+R = {
+    "R_RISCV_32": 1,
+    "R_RISCV_BRANCH": 16,
+    "R_RISCV_JAL": 17,
+    "R_RISCV_CALL": 18,
+    "R_RISCV_CALL_PLT": 19,
+    "R_RISCV_PCREL_HI20": 23,
+    "R_RISCV_PCREL_LO12_I": 24,
+    "R_RISCV_PCREL_LO12_S": 25,
+    "R_RISCV_HI20": 26,
+    "R_RISCV_LO12_I": 27,
+    "R_RISCV_LO12_S": 28,
+    "R_RISCV_RELAX": 51,
+}
+RREV = {v: k for k, v in R.items()}
+
+
+class Elf:
+    def __init__(self, data: bytes):
+        assert data[:4] == b"\x7fELF", "not ELF"
+        assert data[4] == 1 and data[5] == 1, "need ELF32 LE"
+        (self.e_shoff,) = struct.unpack_from("<I", data, 0x20)
+        (self.e_shentsize, self.e_shnum, self.e_shstrndx) = struct.unpack_from("<HHH", data, 0x2E)
+        self.data = data
+        self.sections = []
+        for i in range(self.e_shnum):
+            off = self.e_shoff + i * self.e_shentsize
+            name, typ, flags, addr, offset, size, link, info, align, entsz = \
+                struct.unpack_from("<IIIIIIIIII", data, off)
+            self.sections.append(dict(name=name, type=typ, flags=flags, addr=addr,
+                                      offset=offset, size=size, link=link, info=info,
+                                      align=align, entsize=entsz))
+        shstr = self.sections[self.e_shstrndx]
+        self.shstr = data[shstr["offset"]:shstr["offset"] + shstr["size"]]
+        for s in self.sections:
+            s["sname"] = self._str(self.shstr, s["name"])
+
+    @staticmethod
+    def _str(tab: bytes, off: int) -> str:
+        end = tab.find(b"\x00", off)
+        return tab[off:end].decode()
+
+    def symbols(self):
+        symtab = next(s for s in self.sections if s["sname"] == ".symtab")
+        strtab = self.sections[symtab["link"]]
+        st = self.data[strtab["offset"]:strtab["offset"] + strtab["size"]]
+        syms = []
+        n = symtab["size"] // 16
+        for i in range(n):
+            off = symtab["offset"] + i * 16
+            name, value, size, info, other, shndx = struct.unpack_from("<IIIBBH", self.data, off)
+            syms.append(dict(name=self._str(st, name), value=value, size=size,
+                             info=info, shndx=shndx))
+        return syms
+
+
+def load(obj: bytes, base: int = 0, mem_size: int = 65536,
+         entry_symbol: str = "_start") -> Tuple[List[int], int]:
+    elf = Elf(obj)
+    mem = [0] * mem_size
+
+    # Lay out allocatable progbits/nobits sections (SHF_ALLOC = 0x2).
+    placed: Dict[int, int] = {}   # section index -> load address
+    cursor = base
+    for i, s in enumerate(elf.sections):
+        if not (s["flags"] & 0x2):
+            continue
+        align = max(s["align"], 1)
+        cursor = (cursor + align - 1) & ~(align - 1)
+        placed[i] = cursor
+        if s["type"] == 1:  # PROGBITS
+            body = elf.data[s["offset"]:s["offset"] + s["size"]]
+            for j, byte in enumerate(body):
+                mem[cursor + j] = byte
+        cursor += s["size"]
+
+    syms = elf.symbols()
+
+    def sym_addr(idx: int) -> int:
+        sym = syms[idx]
+        if sym["shndx"] in placed:
+            return placed[sym["shndx"]] + sym["value"]
+        return sym["value"]
+
+    # Apply relocations. PCREL_LO12 references the symbol of its paired HI20
+    # site, so first index HI20/PCREL_HI20 results by their location.
+    hi_at: Dict[int, int] = {}
+    for s in elf.sections:
+        if s["type"] != 4:  # RELA
+            continue
+        target = elf.sections[s["info"]]
+        if s["info"] not in placed:
+            continue
+        tbase = placed[s["info"]]
+        n = s["size"] // 12
+        for i in range(n):
+            off = s["offset"] + i * 12
+            r_offset, r_info, r_addend = struct.unpack_from("<IIi", elf.data, off)
+            r_type = r_info & 0xFF
+            r_sym = r_info >> 8
+            loc = tbase + r_offset
+            if r_type in (R["R_RISCV_PCREL_HI20"], R["R_RISCV_HI20"]):
+                hi_at[loc] = sym_addr(r_sym) + r_addend
+
+    def rd(a):
+        return mem[a] | (mem[a + 1] << 8) | (mem[a + 2] << 16) | (mem[a + 3] << 24)
+
+    def wr(a, v):
+        for k in range(4):
+            mem[a + k] = (v >> (8 * k)) & 0xFF
+
+    for s in elf.sections:
+        if s["type"] != 4 or s["info"] not in placed:
+            continue
+        tbase = placed[s["info"]]
+        n = s["size"] // 12
+        for i in range(n):
+            off = s["offset"] + i * 12
+            r_offset, r_info, r_addend = struct.unpack_from("<IIi", elf.data, off)
+            r_type = r_info & 0xFF
+            r_sym = r_info >> 8
+            loc = tbase + r_offset
+            S = sym_addr(r_sym) + r_addend
+            w = rd(loc)
+            if r_type == R["R_RISCV_32"]:
+                wr(loc, S & 0xFFFFFFFF)
+            elif r_type == R["R_RISCV_HI20"]:
+                wr(loc, (w & 0xFFF) | (((S + 0x800) >> 12) << 12) & 0xFFFFF000)
+            elif r_type == R["R_RISCV_LO12_I"]:
+                wr(loc, (w & 0xFFFFF) | ((S & 0xFFF) << 20))
+            elif r_type == R["R_RISCV_LO12_S"]:
+                imm = S & 0xFFF
+                wr(loc, (w & 0x1FFF07F) | ((imm >> 5) << 25) | ((imm & 31) << 7))
+            elif r_type == R["R_RISCV_PCREL_HI20"]:
+                delta = S - loc
+                wr(loc, (w & 0xFFF) | (((delta + 0x800) >> 12) << 12) & 0xFFFFF000)
+            elif r_type in (R["R_RISCV_PCREL_LO12_I"], R["R_RISCV_PCREL_LO12_S"]):
+                hi = hi_at[sym_addr(r_sym) + r_addend]
+                delta = hi - (sym_addr(r_sym) + r_addend)
+                if r_type == R["R_RISCV_PCREL_LO12_I"]:
+                    wr(loc, (w & 0xFFFFF) | ((delta & 0xFFF) << 20))
+                else:
+                    imm = delta & 0xFFF
+                    wr(loc, (w & 0x1FFF07F) | ((imm >> 5) << 25) | ((imm & 31) << 7))
+            elif r_type in (R["R_RISCV_CALL"], R["R_RISCV_CALL_PLT"]):
+                delta = S - loc
+                hi = (delta + 0x800) >> 12
+                lo = delta - (hi << 12)
+                w2 = rd(loc + 4)
+                wr(loc, (w & 0xFFF) | ((hi << 12) & 0xFFFFF000))
+                wr(loc + 4, (w2 & 0xFFFFF) | ((lo & 0xFFF) << 20))
+            elif r_type == R["R_RISCV_JAL"]:
+                d = S - loc
+                imm = (((d >> 20) & 1) << 31) | (((d >> 1) & 0x3FF) << 21) | \
+                      (((d >> 11) & 1) << 20) | (((d >> 12) & 0xFF) << 12)
+                wr(loc, (w & 0xFFF) | imm)
+            elif r_type == R["R_RISCV_BRANCH"]:
+                d = S - loc
+                imm = (((d >> 12) & 1) << 31) | (((d >> 5) & 0x3F) << 25) | \
+                      (((d >> 1) & 0xF) << 8) | (((d >> 11) & 1) << 7)
+                wr(loc, (w & 0x1FFF07F) | imm)
+            elif r_type == R["R_RISCV_RELAX"]:
+                pass
+            else:
+                raise NotImplementedError(f"reloc {RREV.get(r_type, r_type)}")
+
+    entry = 0
+    for sym in syms:
+        if sym["name"] == entry_symbol:
+            entry = sym_addr(sym["shndx"] and syms.index(sym))
+            entry = placed.get(sym["shndx"], base) + sym["value"]
+            break
+    return mem, entry
+
+# ============================================================================
+# neural_rv32: assembler, reference emulator, threshold runtime
+# ============================================================================
+RV32_MODEL = os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                     "variants", "neural_rv32.safetensors")
+MASK32 = 0xFFFFFFFF
+MMIO_CONSOLE = 0xFF00
+
+
+def fcvt_s_w_ref(x: int) -> int:
+    """Signed int32 -> float32 word, round-to-nearest-even (host float()
+    then pack to float32 is exact RNE for the int32 range)."""
+    xi = x if (x & MASK32) < 0x80000000 else (x & MASK32) - (1 << 32)
+    return struct.unpack("<I", struct.pack("<f", float(xi)))[0]
+
+
+def fcvt_w_s_ref(w: int) -> int:
+    """float32 word -> signed int32, round toward zero (RISC-V rtz),
+    saturating; NaN -> INT_MAX, infinities -> INT_MIN/INT_MAX."""
+    s, e, f = (w >> 31) & 1, (w >> 23) & 0xFF, w & 0x7FFFFF
+    if e == 0xFF:
+        return 0x7FFFFFFF if f else (0x80000000 if s else 0x7FFFFFFF)
+    E = e - 127
+    if E < 0:
+        return 0
+    mant = (1 << 23) | f
+    if E >= 31:
+        if s and E == 31 and f == 0:
+            return 0x80000000
+        return 0x80000000 if s else 0x7FFFFFFF
+    mag = (mant << (E - 23)) if E >= 23 else (mant >> (23 - E))
+    return ((-mag) & MASK32) if s else (mag & MASK32)
+
+
+def sext(v: int, bits: int) -> int:
+    v &= (1 << bits) - 1
+    return v - (1 << bits) if v & (1 << (bits - 1)) else v
+
+
+def u32(v: int) -> int:
+    return v & MASK32
+
+
+# =============================================================================
+# Assembler
+# =============================================================================
+
+R_FUNCT = {"add": (0, 0), "sub": (0, 32), "sll": (1, 0), "slt": (2, 0),
+           "sltu": (3, 0), "xor": (4, 0), "srl": (5, 0), "sra": (5, 32),
+           "or": (6, 0), "and": (7, 0),
+           "mul": (0, 1), "mulh": (1, 1), "mulhsu": (2, 1), "mulhu": (3, 1),
+           "div": (4, 1), "divu": (5, 1), "rem": (6, 1), "remu": (7, 1)}
+I_FUNCT = {"addi": 0, "slti": 2, "sltiu": 3, "xori": 4, "ori": 6, "andi": 7}
+B_FUNCT = {"beq": 0, "bne": 1, "blt": 4, "bge": 5, "bltu": 6, "bgeu": 7}
+L_FUNCT = {"lb": 0, "lh": 1, "lw": 2, "lbu": 4, "lhu": 5}
+S_FUNCT = {"sb": 0, "sh": 1, "sw": 2}
+
+
+class Asm:
+    def __init__(self):
+        self.words: List = []
+        self.labels: Dict[str, int] = {}
+        self.data_here = None
+
+    def label(self, name):
+        self.labels[name] = len(self.words) * 4
+
+    def emit(self, *parts):
+        self.words.append(parts)
+
+    def __getattr__(self, name):
+        if name.startswith("__"):
+            raise AttributeError(name)
+        mn = name.rstrip("_").replace("_", ".")
+        def f(*args):
+            self.emit(mn, *args)
+        return f
+
+    def word(self, v):
+        self.emit(".word", v)
+
+    def _enc(self, parts, pc):
+        mn = parts[0]
+        a = parts[1:]
+
+        def imm(x, pcrel=False):
+            if isinstance(x, str):
+                t = self.labels[x]
+                return t - pc if pcrel else t
+            return x
+
+        if mn == ".word":
+            return imm(a[0]) & MASK32
+        if mn in R_FUNCT:
+            f3, f7 = R_FUNCT[mn]
+            return (f7 << 25) | (a[2] << 20) | (a[1] << 15) | (f3 << 12) | (a[0] << 7) | 0x33
+        if mn in I_FUNCT:
+            return ((imm(a[2]) & 0xFFF) << 20) | (a[1] << 15) | (I_FUNCT[mn] << 12) | (a[0] << 7) | 0x13
+        if mn in ("slli", "srli", "srai"):
+            f7 = 32 if mn == "srai" else 0
+            f3 = 1 if mn == "slli" else 5
+            return (f7 << 25) | ((a[2] & 31) << 20) | (a[1] << 15) | (f3 << 12) | (a[0] << 7) | 0x13
+        if mn in L_FUNCT:
+            return ((imm(a[2]) & 0xFFF) << 20) | (a[1] << 15) | (L_FUNCT[mn] << 12) | (a[0] << 7) | 0x03
+        if mn in S_FUNCT:
+            i = imm(a[2]) & 0xFFF
+            return ((i >> 5) << 25) | (a[0] << 20) | (a[1] << 15) | (S_FUNCT[mn] << 12) | ((i & 31) << 7) | 0x23
+        if mn in B_FUNCT:
+            i = imm(a[2], pcrel=True) & 0x1FFF
+            return (((i >> 12) & 1) << 31) | (((i >> 5) & 0x3F) << 25) | (a[1] << 20) | \
+                   (a[0] << 15) | (B_FUNCT[mn] << 12) | (((i >> 1) & 0xF) << 8) | (((i >> 11) & 1) << 7) | 0x63
+        if mn == "lui":
+            return ((imm(a[1]) & 0xFFFFF) << 12) | (a[0] << 7) | 0x37
+        if mn == "auipc":
+            return ((imm(a[1]) & 0xFFFFF) << 12) | (a[0] << 7) | 0x17
+        if mn == "jal":
+            i = imm(a[1], pcrel=True) & 0x1FFFFF
+            return (((i >> 20) & 1) << 31) | (((i >> 1) & 0x3FF) << 21) | (((i >> 11) & 1) << 20) | \
+                   (((i >> 12) & 0xFF) << 12) | (a[0] << 7) | 0x6F
+        if mn == "jalr":
+            return ((imm(a[2]) & 0xFFF) << 20) | (a[1] << 15) | (a[0] << 7) | 0x67
+        if mn == "flw":
+            return ((imm(a[2]) & 0xFFF) << 20) | (a[1] << 15) | (2 << 12) | (a[0] << 7) | 0x07
+        if mn == "fsw":
+            i = imm(a[2]) & 0xFFF
+            return ((i >> 5) << 25) | (a[0] << 20) | (a[1] << 15) | (2 << 12) | ((i & 31) << 7) | 0x27
+        if mn in ("fadd.s", "fsub.s", "fmul.s", "fdiv.s"):
+            f7 = {"fadd.s": 0, "fsub.s": 4, "fmul.s": 8, "fdiv.s": 12}[mn]
+            return (f7 << 25) | (a[2] << 20) | (a[1] << 15) | (a[0] << 7) | 0x53
+        if mn in ("fle.s", "flt.s", "feq.s"):
+            f3 = {"fle.s": 0, "flt.s": 1, "feq.s": 2}[mn]
+            return (0x50 << 25) | (a[2] << 20) | (a[1] << 15) | (f3 << 12) | (a[0] << 7) | 0x53
+        if mn == "fmv.w.x":
+            return (0x78 << 25) | (a[1] << 15) | (a[0] << 7) | 0x53
+        if mn == "fmv.x.w":
+            return (0x70 << 25) | (a[1] << 15) | (a[0] << 7) | 0x53
+        if mn == "fcvt.w.s":
+            return (0x60 << 25) | (a[1] << 15) | (1 << 12) | (a[0] << 7) | 0x53  # rm=rtz
+        if mn == "fcvt.s.w":
+            return (0x68 << 25) | (a[1] << 15) | (a[0] << 7) | 0x53
+        if mn in ("fsgnj.s", "fsgnjn.s", "fsgnjx.s"):
+            f3 = {"fsgnj.s": 0, "fsgnjn.s": 1, "fsgnjx.s": 2}[mn]
+            return (0x10 << 25) | (a[2] << 20) | (a[1] << 15) | (f3 << 12) | (a[0] << 7) | 0x53
+        if mn == "neur":
+            return (a[2] << 20) | (a[1] << 15) | (a[0] << 7) | 0x0B
+        if mn == "ecall":
+            return 0x73
+        if mn == "fence":
+            return 0x0F
+        raise ValueError(f"unknown mnemonic {mn}")
+
+    def assemble(self, mem_size=65536) -> List[int]:
+        mem = [0] * mem_size
+        for idx, parts in enumerate(self.words):
+            w = self._enc(parts, idx * 4)
+            for b in range(4):
+                mem[idx * 4 + b] = (w >> (8 * b)) & 0xFF   # little-endian
+        return mem
+
+
+# =============================================================================
+# Reference emulator (architectural contract)
+# =============================================================================
+
+class RefRV32:
+    @staticmethod
+    def step(s):
+        if s["halted"]:
+            return s
+        s = dict(s, mem=s["mem"][:], regs=s["regs"][:], fregs=s["fregs"][:],
+                 console=s["console"])
+        pc = s["pc"]
+        w = RefRV32._fetch(s, pc)
+        RefRV32._exec(s, pc, w)
+        return s
+
+    @staticmethod
+    def _fetch(s, pc):
+        m = s["mem"]
+        return m[pc] | (m[pc + 1] << 8) | (m[pc + 2] << 16) | (m[pc + 3] << 24)
+
+    @staticmethod
+    def _load(s, addr, width, signed):
+        v = 0
+        for b in range(width):
+            v |= s["mem"][(addr + b) & 0xFFFF] << (8 * b)
+        return u32(sext(v, width * 8)) if signed else v
+
+    @staticmethod
+    def _store(s, addr, val, width):
+        for b in range(width):
+            s["mem"][(addr + b) & 0xFFFF] = (val >> (8 * b)) & 0xFF
+        if (addr & 0xFFFF) == MMIO_CONSOLE:
+            s["console"] += chr(val & 0xFF)
+
+    @staticmethod
+    def _exec(s, pc, w):
+        op = w & 0x7F
+        rd = (w >> 7) & 31
+        f3 = (w >> 12) & 7
+        rs1 = (w >> 15) & 31
+        rs2 = (w >> 20) & 31
+        f7 = w >> 25
+        R = s["regs"]
+        FR = s["fregs"]
+        a, b = R[rs1], R[rs2]
+        nxt = (pc + 4) & MASK32
+
+        def wr(v):
+            if rd:
+                R[rd] = u32(v)
+
+        if op == 0x33:  # OP (+ M)
+            if f7 == 1:
+                sa, sb = sext(a, 32), sext(b, 32)
+                if f3 == 0: wr(a * b)
+                elif f3 == 1: wr((sa * sb) >> 32)
+                elif f3 == 2: wr((sa * b) >> 32)
+                elif f3 == 3: wr((a * b) >> 32)
+                elif f3 == 4: wr(MASK32 if b == 0 else (0x80000000 if (sa, sb) == (-2**31, -1) else int(abs(sa) // abs(sb)) * (1 if (sa < 0) == (sb < 0) else -1)))
+                elif f3 == 5: wr(MASK32 if b == 0 else a // b)
+                elif f3 == 6: wr(sa if b == 0 else (0 if (sa, sb) == (-2**31, -1) else sa - (int(abs(sa) // abs(sb)) * (1 if (sa < 0) == (sb < 0) else -1)) * sb))
+                elif f3 == 7: wr(a if b == 0 else a % b)
+            else:
+                sh = b & 31
+                if f3 == 0: wr(a - b if f7 == 32 else a + b)
+                elif f3 == 1: wr(a << sh)
+                elif f3 == 2: wr(1 if sext(a, 32) < sext(b, 32) else 0)
+                elif f3 == 3: wr(1 if a < b else 0)
+                elif f3 == 4: wr(a ^ b)
+                elif f3 == 5: wr(u32(sext(a, 32) >> sh) if f7 == 32 else a >> sh)
+                elif f3 == 6: wr(a | b)
+                elif f3 == 7: wr(a & b)
+        elif op == 0x13:  # OP-IMM
+            imm = sext(w >> 20, 12)
+            sh = (w >> 20) & 31
+            if f3 == 0: wr(a + imm)
+            elif f3 == 1: wr(a << sh)
+            elif f3 == 2: wr(1 if sext(a, 32) < imm else 0)
+            elif f3 == 3: wr(1 if a < u32(imm) else 0)
+            elif f3 == 4: wr(a ^ u32(imm))
+            elif f3 == 5: wr(u32(sext(a, 32) >> sh) if f7 == 32 else a >> sh)
+            elif f3 == 6: wr(a | u32(imm))
+            elif f3 == 7: wr(a & u32(imm))
+        elif op == 0x37: wr(w & 0xFFFFF000)              # LUI
+        elif op == 0x17: wr(pc + (w & 0xFFFFF000))       # AUIPC
+        elif op == 0x6F:                                  # JAL
+            imm = sext((((w >> 31) & 1) << 20) | (((w >> 12) & 0xFF) << 12)
+                       | (((w >> 20) & 1) << 11) | (((w >> 21) & 0x3FF) << 1), 21)
+            wr(nxt)
+            nxt = u32(pc + imm)
+        elif op == 0x67:                                  # JALR
+            t = u32(a + sext(w >> 20, 12)) & ~1
+            wr(nxt)
+            nxt = t
+        elif op == 0x63:                                  # BRANCH
+            imm = sext((((w >> 31) & 1) << 12) | (((w >> 7) & 1) << 11)
+                       | (((w >> 25) & 0x3F) << 5) | (((w >> 8) & 0xF) << 1), 13)
+            sa, sb = sext(a, 32), sext(b, 32)
+            take = [a == b, a != b, None, None, sa < sb, sa >= sb, a < b, a >= b][f3]
+            if take:
+                nxt = u32(pc + imm)
+        elif op == 0x03:                                  # LOAD
+            addr = u32(a + sext(w >> 20, 12)) & 0xFFFF
+            width = [1, 2, 4, 0, 1, 2][f3]
+            wr(RefRV32._load(s, addr, width, f3 < 3))
+        elif op == 0x23:                                  # STORE
+            imm = sext(((w >> 25) << 5) | ((w >> 7) & 31), 12)
+            addr = u32(a + imm) & 0xFFFF
+            RefRV32._store(s, addr, b, [1, 2, 4][f3])
+        elif op == 0x07:                                  # FLW
+            addr = u32(a + sext(w >> 20, 12)) & 0xFFFF
+            FR[rd] = RefRV32._load(s, addr, 4, False)
+        elif op == 0x27:                                  # FSW
+            imm = sext(((w >> 25) << 5) | ((w >> 7) & 31), 12)
+            addr = u32(a + imm) & 0xFFFF
+            RefRV32._store(s, addr, FR[rs2], 4)
+        elif op == 0x53:                                  # OP-FP
+            from eval import float_add_oracle, float_mul_oracle, float_div_oracle
+            fa, fb = FR[rs1], FR[rs2]
+            if f7 == 0x00: FR[rd] = float_add_oracle(fa, fb, 8, 23)
+            elif f7 == 0x04: FR[rd] = float_add_oracle(fa, fb ^ 0x80000000, 8, 23)
+            elif f7 == 0x08: FR[rd] = float_mul_oracle(fa, fb, 8, 23)
+            elif f7 == 0x0C: FR[rd] = float_div_oracle(fa, fb, 8, 23)
+            elif f7 == 0x10:
+                sa, sb = (fa >> 31) & 1, (fb >> 31) & 1
+                so = [sb, 1 - sb, sa ^ sb][f3]
+                FR[rd] = (fa & 0x7FFFFFFF) | (so << 31)
+            elif f7 == 0x50:
+                from eval import float_bits_to_value
+                x, y = float_bits_to_value(fa, 8, 23), float_bits_to_value(fb, 8, 23)
+                wr(1 if [x <= y, x < y, x == y][f3] else 0)
+            elif f7 == 0x60: wr(fcvt_w_s_ref(FR[rs1]))     # FCVT.W.S
+            elif f7 == 0x68: FR[rd] = fcvt_s_w_ref(R[rs1]) # FCVT.S.W
+            elif f7 == 0x78: FR[rd] = a
+            elif f7 == 0x70: wr(FR[rs1])
+        elif op == 0x0B:                                  # NEUR (custom-0)
+            x = a & 0xFF
+            pos, neg = b & 0xFF, (b >> 8) & 0xFF
+            bias = sext(b >> 16, 5)
+            acc = bin(x & pos).count("1") - bin(x & neg).count("1") + bias
+            wr(1 if acc >= 0 else 0)
+        elif op == 0x73:
+            s["halted"] = True
+        elif op == 0x0F:
+            pass
+        else:
+            raise ValueError(f"illegal instruction {w:#010x} at {pc:#06x}")
+        s["pc"] = nxt
+
+    @staticmethod
+    def run(state, max_cycles=10000):
+        s = state
+        n = 0
+        while not s["halted"] and n < max_cycles:
+            s = RefRV32.step(s)
+            n += 1
+        return s, n
+
+
+def rv_state(mem):
+    return {"pc": 0, "regs": [0] * 32, "fregs": [0] * 32, "mem": mem[:],
+            "halted": False, "console": ""}
+
+
+# =============================================================================
+# Threshold runtime
+# =============================================================================
+
+PAIRABLE = (0x33, 0x13, 0x37, 0x17)
+
+
+class Rv32ThresholdCPU:
+    def __init__(self, path=RV32_MODEL):
+        self.T = {}
+        with safe_open(path, framework="pt") as f:
+            for name in f.keys():
+                self.T[name] = f.get_tensor(name).float()
+        self.registry = load_metadata(path)["signal_registry"]
+        self.alu = GenericThresholdALU(self.T, 32)
+        self.fpu = {op: NetlistEvaluator(self.T, self.registry, f"float32.{op}")
+                    for op in ("add", "mul", "div", "cmp")}
+        self.pairs_issued = 0
+
+    # ---- gate helpers -------------------------------------------------
+    def _g(self, name, inputs):
+        return self.alu._g(name, inputs)
+
+    def _not1(self, x):
+        return self._g("alu.alu32bit.not.bit0", [float(x)])
+
+    def _bitwise(self, opname, a, b):
+        ab = int_to_bits(a, 32)
+        bb = int_to_bits(b, 32)
+        out = []
+        for k in range(32):
+            if opname == "xor":
+                p = f"alu.alu32bit.xor.bit{k}"
+                h1 = self._g(f"{p}.layer1.or", [ab[k], bb[k]])
+                h2 = self._g(f"{p}.layer1.nand", [ab[k], bb[k]])
+                out.append(self._g(f"{p}.layer2", [h1, h2]))
+            else:
+                out.append(self._g(f"alu.alu32bit.{opname}.bit{k}", [ab[k], bb[k]]))
+        return bits_to_int(out)
+
+    def _not32(self, a):
+        ab = int_to_bits(a, 32)
+        return bits_to_int([self._g(f"alu.alu32bit.not.bit{k}", [ab[k]]) for k in range(32)])
+
+    def _barrel_left(self, val, sh):
+        layer_in = int_to_bits(val, 32)
+        for layer in range(5):
+            amount = 1 << (4 - layer)
+            sel = (sh >> (4 - layer)) & 1
+            nxt = []
+            for bit in range(32):
+                p = f"combinational.barrelshifter32.layer{layer}.bit{bit}"
+                ns = self._g(f"{p}.not_sel", [float(sel)])
+                src = layer_in[bit + amount] if bit + amount < 32 else 0
+                a_ = self._g(f"{p}.and_a", [layer_in[bit], ns])
+                b_ = self._g(f"{p}.and_b", [float(src), float(sel)])
+                nxt.append(self._g(f"{p}.or", [a_, b_]))
+            layer_in = nxt
+        return bits_to_int(layer_in)
+
+    def sll(self, a, sh):
+        return self._barrel_left(a, sh & 31)
+
+    def srl(self, a, sh):
+        rev = int("{:032b}".format(u32(a))[::-1], 2)
+        return int("{:032b}".format(self._barrel_left(rev, sh & 31))[::-1], 2)
+
+    def sra(self, a, sh):
+        sign = (a >> 31) & 1
+        x1 = self.srl(a, sh)
+        x2 = self._not32(self.srl(self._not32(a), sh))
+        o1, o2 = int_to_bits(x1, 32), int_to_bits(x2, 32)
+        out = []
+        for k in range(32):
+            p = f"rv32.sra_mux.bit{k}"
+            ns = self._g(f"{p}.not_sel", [float(sign)])
+            a_ = self._g(f"{p}.and_a", [o1[k], ns])
+            b_ = self._g(f"{p}.and_b", [o2[k], float(sign)])
+            out.append(self._g(f"{p}.or", [a_, b_]))
+        return bits_to_int(out)
+
+    def add32(self, a, b):
+        return self.alu.add_n(a, b, 32)[0]
+
+    def sub32(self, a, b):
+        return self.alu.sub_n(a, b, 32)[0]
+
+    def ucmp(self, a, b, kind):
+        return self.alu.cmp_n(a, b, kind, 32)
+
+    def scmp_lt(self, a, b):
+        fa = (a & 0x7FFFFFFF) | (self._not1((a >> 31) & 1) << 31)
+        fb = (b & 0x7FFFFFFF) | (self._not1((b >> 31) & 1) << 31)
+        return self.ucmp(fa, fb, "lessthan")
+
+    def mulhu(self, a, b):
+        """Full 64-bit product high word via pp gates + rv32.mulacc chains."""
+        ab = int_to_bits(a, 32)
+        bb = int_to_bits(b, 32)
+
+        def pp(i_lsb, j_lsb):  # LSB-first indices onto MSB-first gate names
+            return self._g(f"alu.alu32bit.mul.pp.a{31 - i_lsb}b{31 - j_lsb}",
+                           [ab[31 - i_lsb], bb[31 - j_lsb]])
+
+        acc = [pp(k, 0) if k <= 31 else 0.0 for k in range(64)]
+        for t in range(31):
+            row = t + 1
+            carry = 0.0
+            nxt = []
+            for k in range(64):
+                b_in = pp(k - row, row) if row <= k <= row + 31 else 0.0
+                s, carry = self.alu._fa(f"rv32.mulacc.s{t}.fa{k}", acc[k], b_in, carry)
+                nxt.append(s)
+            acc = nxt
+        return bits_to_int(list(reversed([int(x) for x in acc[32:]])))
+
+    def _cond_mask(self, val, cond_bit):
+        vb = int_to_bits(val, 32)
+        return bits_to_int([self._g(f"alu.alu32bit.and.bit{k}", [vb[k], float(cond_bit)])
+                            for k in range(32)])
+
+    def mulh(self, a, b, sign_a, sign_b):
+        h = self.mulhu(a, b)
+        if sign_a:
+            h = self.sub32(h, self._cond_mask(b, (a >> 31) & 1))
+        if sign_b:
+            h = self.sub32(h, self._cond_mask(a, (b >> 31) & 1))
+        return h
+
+    def divu(self, a, b):
+        if b == 0:
+            return MASK32, a
+        a_bits = int_to_bits(a, 32)
+        q = 0
+        rem = 0
+        for stage in range(32):
+            rem = ((rem << 1) | a_bits[stage]) & MASK32
+            prefix = f"alu.alu32bit.div.stage{stage}"
+            finals = {"gt": f"{prefix}.cmp_bc.gt", "lt": f"{prefix}.cmp_bc.lt",
+                      "eq": f"{prefix}.cmp_bc.eq", "ge": f"{prefix}.cmp",
+                      "le": f"{prefix}.cmp_bc.le"}
+            ge = self.alu._cmp_bit_cascade(f"{prefix}.cmp_bc", finals,
+                                           int_to_bits(rem, 32), int_to_bits(b, 32), "ge")
+            if ge:
+                rem = self.sub32(rem, b)
+                q = (q << 1) | 1
+            else:
+                q <<= 1
+        return u32(q), rem
+
+    def neg32(self, a):
+        ab = int_to_bits(a, 32)
+        nb = [self._g(f"alu.alu32bit.neg.not.bit{k}", [ab[k]]) for k in range(32)]
+        carry = 1.0
+        out = [0] * 32
+        for k in range(31, -1, -1):
+            p = f"alu.alu32bit.neg.inc.bit{k}"
+            h1 = self._g(f"{p}.xor.layer1.or", [nb[k], carry])
+            h2 = self._g(f"{p}.xor.layer1.nand", [nb[k], carry])
+            out[k] = self._g(f"{p}.xor.layer2", [h1, h2])
+            carry = self._g(f"{p}.carry", [nb[k], carry])
+        return bits_to_int(out)
+
+    def divs(self, a, b):
+        if b == 0:
+            return MASK32, a
+        if a == 0x80000000 and b == MASK32:
+            return 0x80000000, 0
+        na, nb_ = (a >> 31) & 1, (b >> 31) & 1
+        ua = self.neg32(a) if na else a
+        ub = self.neg32(b) if nb_ else b
+        q, r = self.divu(ua, ub)
+        if na != nb_:
+            q = self.neg32(q)
+        if na:
+            r = self.neg32(r)
+        return q, r
+
+    def priority_encode(self, v):
+        """MSB-first position (0..31) of the highest set bit, via the
+        combinational.priorityencoder32 gates. v must be nonzero."""
+        vb = int_to_bits(v, 32)  # MSB-first, bit index 0 = MSB
+        any_higher = [0]
+        for pos in range(1, 32):
+            any_higher.append(self._g(f"combinational.priorityencoder32.any_higher{pos}",
+                                      [float(vb[i]) for i in range(pos)]))
+        is_high = [float(vb[0])]
+        for pos in range(1, 32):
+            nh = self._g(f"combinational.priorityencoder32.is_highest{pos}.not_higher",
+                         [float(any_higher[pos])])
+            is_high.append(self._g(f"combinational.priorityencoder32.is_highest{pos}.and",
+                                   [float(vb[pos]), nh]))
+        out = 0
+        for bit in range(5):
+            rel = [is_high[pos] for pos in range(32) if (pos >> bit) & 1]
+            out |= self._g(f"combinational.priorityencoder32.out{bit}", [float(x) for x in rel]) << bit
+        return out
+
+    def fcvt_s_w(self, x):
+        """Signed int32 -> float32, round-to-nearest-even. Sign, leading-one
+        detect (priority encoder), left-normalize (barrel shifter), round."""
+        if x == 0:
+            return 0
+        s = (x >> 31) & 1
+        u = self.neg32(x) if s else x
+        p = self.priority_encode(u)                 # leading one at MSB-first pos p
+        shifted = self.sll(u, p)                     # leading one now at bit 31
+        exp = 127 + (31 - p)
+        frac = (shifted >> 8) & 0x7FFFFF
+        guard = (shifted >> 7) & 1
+        below = shifted & 0x7F
+        if guard and ((frac & 1) or below):
+            frac += 1
+            if frac > 0x7FFFFF:
+                frac = 0
+                exp += 1
+        return (s << 31) | (exp << 23) | frac
+
+    def fcvt_w_s(self, w):
+        """float32 -> signed int32, round toward zero (RISC-V rtz), saturating.
+        NaN -> INT_MAX; overflow saturates to INT_MIN/INT_MAX."""
+        s, e, f = (w >> 31) & 1, (w >> 23) & 0xFF, w & 0x7FFFFF
+        if e == 0xFF:
+            if f:
+                return 0x7FFFFFFF                    # NaN
+            return 0x80000000 if s else 0x7FFFFFFF   # +-inf
+        E = e - 127
+        if E < 0:
+            return 0
+        mant = (1 << 23) | f
+        if E >= 31:
+            if s and E == 31 and f == 0:
+                return 0x80000000                    # exactly -2^31
+            return 0x80000000 if s else 0x7FFFFFFF
+        if E >= 23:
+            mag = self.sll(mant, E - 23)
+        else:
+            mag = self.srl(mant, 23 - E)
+        return self.neg32(mag) if s else (mag & MASK32)
+
+    def neur(self, x_reg, w_reg):
+        x = int_to_bits(x_reg & 0xFF, 8)      # MSB-first bits of low byte
+        pos = int_to_bits(w_reg & 0xFF, 8)
+        neg = int_to_bits((w_reg >> 8) & 0xFF, 8)
+        p_bits = [self._g(f"rv32.neur.pos.bit{i}", [x[7 - i], pos[7 - i]]) for i in range(8)]
+        n_bits = [self._g(f"rv32.neur.neg.bit{i}", [x[7 - i], neg[7 - i]]) for i in range(8)]
+
+        def pcount(side, bits8):
+            fa = lambda t, a_, b_, c_: self.alu._fa(f"rv32.neur.{side}.fa{t}", a_, b_, c_)
+            s0, c0 = fa(0, bits8[0], bits8[1], bits8[2])
+            s1, c1 = fa(1, bits8[3], bits8[4], bits8[5])
+            s2, c2 = fa(2, bits8[6], bits8[7], 0.0)
+            S, C2 = fa(3, s0, s1, s2)
+            T, C4 = fa(4, c0, c1, c2)
+            U, C5 = fa(5, C2, T, 0.0)
+            V, W = fa(6, C4, C5, 0.0)
+            return [S, U, V, W]  # LSB-first count bits
+
+        P = pcount("pcnt", p_bits)
+        N = pcount("ncnt", n_bits)
+        nnot = [self._g(f"rv32.neur.nnot.bit{k}", [N[k] if k < 4 else 0.0]) for k in range(6)]
+        carry = 1.0
+        diff = []
+        for k in range(6):
+            a_ = P[k] if k < 4 else 0.0
+            s, carry = self.alu._fa(f"rv32.neur.diff.fa{k}", a_, nnot[k], carry)
+            diff.append(s)
+        bias = (w_reg >> 16) & 0x1F
+        bias6 = [(bias >> k) & 1 if k < 4 else (bias >> 4) & 1 for k in range(6)]  # sext5
+        carry = 0.0
+        act = []
+        for k in range(6):
+            s, carry = self.alu._fa(f"rv32.neur.act.fa{k}", diff[k], float(bias6[k]), carry)
+            act.append(s)
+        return int(self._g("rv32.neur.out", [act[5]]))
+
+    def hazard_eq(self, pair, i, j):
+        ib = int_to_bits(i, 5)
+        jb = int_to_bits(j, 5)
+        eqs = []
+        for k in range(5):
+            p = f"rv32.hazard.{pair}.bit{k}"
+            h_and = self._g(f"{p}.eq.layer1.and", [ib[k], jb[k]])
+            h_nor = self._g(f"{p}.eq.layer1.nor", [ib[k], jb[k]])
+            eqs.append(self._g(f"{p}.eq", [h_and, h_nor]))
+        return int(self._g(f"rv32.hazard.{pair}.all", eqs))
+
+    # ---- memory through the packed circuits ---------------------------
+    def _addr_decode(self, addr):
+        bits = torch.tensor(int_to_bits(addr, 16), dtype=torch.float32)
+        return heaviside((self.T["memory.addr_decode.weight"] * bits).sum(dim=1)
+                         + self.T["memory.addr_decode.bias"])
+
+    def mem_read_byte(self, mem_bits, addr):
+        sel = self._addr_decode(addr & 0xFFFF)
+        and_w = self.T["memory.read.and.weight"]
+        and_b = self.T["memory.read.and.bias"]
+        or_w = self.T["memory.read.or.weight"]
+        or_b = self.T["memory.read.or.bias"]
+        v = 0
+        for bit in range(8):
+            inp = torch.stack([mem_bits[:, bit], sel], dim=1)
+            and_out = heaviside((inp * and_w[bit]).sum(dim=1) + and_b[bit])
+            v = (v << 1) | int(heaviside((and_out * or_w[bit]).sum() + or_b[bit]).item())
+        return v
+
+    def mem_write_byte(self, mem_bits, addr, value):
+        sel = self._addr_decode(addr & 0xFFFF)
+        data_bits = torch.tensor(int_to_bits(value & 0xFF, 8), dtype=torch.float32)
+        we = torch.ones_like(sel)
+        write_sel = heaviside((torch.stack([sel, we], dim=1)
+                               * self.T["memory.write.sel.weight"]).sum(dim=1)
+                              + self.T["memory.write.sel.bias"])
+        nsel = heaviside(write_sel * self.T["memory.write.nsel.weight"].squeeze(1)
+                         + self.T["memory.write.nsel.bias"])
+        for bit in range(8):
+            old = mem_bits[:, bit]
+            dbit = data_bits[bit].expand(65536)
+            a_old = heaviside((torch.stack([old, nsel], dim=1)
+                               * self.T["memory.write.and_old.weight"][:, bit]).sum(dim=1)
+                              + self.T["memory.write.and_old.bias"][:, bit])
+            a_new = heaviside((torch.stack([dbit, write_sel], dim=1)
+                               * self.T["memory.write.and_new.weight"][:, bit]).sum(dim=1)
+                              + self.T["memory.write.and_new.bias"][:, bit])
+            mem_bits[:, bit] = heaviside((torch.stack([a_old, a_new], dim=1)
+                                          * self.T["memory.write.or.weight"][:, bit]).sum(dim=1)
+                                         + self.T["memory.write.or.bias"][:, bit])
+
+    # ---- execution ----------------------------------------------------
+    def _fetch32(self, mem_bits, pc):
+        return sum(self.mem_read_byte(mem_bits, pc + b) << (8 * b) for b in range(4))
+
+    def _load(self, s, addr, width, signed):
+        v = sum(self.mem_read_byte(s["_mem_bits"], (addr + b) & 0xFFFF) << (8 * b)
+                for b in range(width))
+        return u32(sext(v, width * 8)) if signed else v
+
+    def _store(self, s, addr, val, width):
+        for b in range(width):
+            byte = (val >> (8 * b)) & 0xFF
+            self.mem_write_byte(s["_mem_bits"], (addr + b) & 0xFFFF, byte)
+            s["mem"][(addr + b) & 0xFFFF] = byte
+        if (addr & 0xFFFF) == MMIO_CONSOLE:
+            s["console"] += chr(val & 0xFF)
+
+    def _exec_alu_class(self, s, pc, w):
+        """Execute one OP / OP-IMM / LUI / AUIPC instruction; returns (rd, value)."""
+        op = w & 0x7F
+        rd = (w >> 7) & 31
+        f3 = (w >> 12) & 7
+        rs1 = (w >> 15) & 31
+        rs2 = (w >> 20) & 31
+        f7 = w >> 25
+        R = s["regs"]
+        a = R[rs1]
+        if op == 0x37:
+            return rd, w & 0xFFFFF000
+        if op == 0x17:
+            return rd, self.add32(pc, w & 0xFFFFF000)
+        if op == 0x13:
+            b = u32(sext(w >> 20, 12))
+            sh = (w >> 20) & 31
+            imm_mode = True
+        else:
+            b = R[rs2]
+            sh = b & 31
+            imm_mode = False
+        if f3 == 0:
+            if not imm_mode and f7 == 32:
+                return rd, self.sub32(a, b)
+            return rd, self.add32(a, b)
+        if f3 == 1:
+            return rd, self.sll(a, sh)
+        if f3 == 2:
+            return rd, self.scmp_lt(a, b)
+        if f3 == 3:
+            return rd, self.ucmp(a, b, "lessthan")
+        if f3 == 4:
+            return rd, self._bitwise("xor", a, b)
+        if f3 == 5:
+            return rd, (self.sra(a, sh) if f7 == 32 else self.srl(a, sh))
+        if f3 == 6:
+            return rd, self._bitwise("or", a, b)
+        return rd, self._bitwise("and", a, b)
+
+    def step(self, s):
+        if s["halted"]:
+            return s
+        s = dict(s, mem=s["mem"][:], regs=s["regs"][:], fregs=s["fregs"][:],
+                 console=s["console"], _mem_bits=s["_mem_bits"].clone())
+        pc = s["pc"]
+        w = self._fetch32(s["_mem_bits"], pc)
+        op = w & 0x7F
+
+        def plain_alu(word):
+            return (word & 0x7F) in PAIRABLE and not \
+                ((word & 0x7F) == 0x33 and (word >> 25) == 1)
+
+        # Dual issue: both slots plain ALU-class and hazard-free by gates.
+        if plain_alu(w):
+            w2 = self._fetch32(s["_mem_bits"], (pc + 4) & 0xFFFF)
+            if plain_alu(w2):
+                rd1 = (w >> 7) & 31
+                rs1b = (w2 >> 15) & 31
+                rs2b = (w2 >> 20) & 31
+                rd2 = (w2 >> 7) & 31
+                uses_rs1 = (w2 & 0x7F) in (0x33, 0x13)
+                uses_rs2 = (w2 & 0x7F) == 0x33
+                raw = (uses_rs1 and self.hazard_eq("rd1_rs1b", rd1, rs1b)) or \
+                      (uses_rs2 and self.hazard_eq("rd1_rs2b", rd1, rs2b))
+                waw = self.hazard_eq("rd1_rd2", rd1, rd2)
+                if rd1 != 0 and not raw and not waw:
+                    d1, v1 = self._exec_alu_class(s, pc, w)
+                    d2, v2 = self._exec_alu_class(s, (pc + 4) & MASK32, w2)
+                    if d1:
+                        s["regs"][d1] = u32(v1)
+                    if d2:
+                        s["regs"][d2] = u32(v2)
+                    s["pc"] = (pc + 8) & MASK32
+                    self.pairs_issued += 1
+                    return s
+
+        rd = (w >> 7) & 31
+        f3 = (w >> 12) & 7
+        rs1 = (w >> 15) & 31
+        rs2 = (w >> 20) & 31
+        f7 = w >> 25
+        R = s["regs"]
+        FR = s["fregs"]
+        a, b = R[rs1], R[rs2]
+        nxt = (pc + 4) & MASK32
+
+        def wr(v):
+            if rd:
+                R[rd] = u32(v)
+
+        if plain_alu(w):
+            d, v = self._exec_alu_class(s, pc, w)
+            if d:
+                R[d] = u32(v)
+        if op == 0x33 and f7 == 1:
+            if f3 == 0:
+                wr(self.alu.mul_n(a, b, 32))
+            elif f3 == 1:
+                wr(self.mulh(a, b, True, True))
+            elif f3 == 2:
+                wr(self.mulh(a, b, True, False))
+            elif f3 == 3:
+                wr(self.mulhu(a, b))
+            elif f3 == 4:
+                wr(self.divs(a, b)[0])
+            elif f3 == 5:
+                wr(self.divu(a, b)[0])
+            elif f3 == 6:
+                wr(self.divs(a, b)[1])
+            elif f3 == 7:
+                wr(self.divu(a, b)[1])
+        elif op == 0x6F:
+            imm = sext((((w >> 31) & 1) << 20) | (((w >> 12) & 0xFF) << 12)
+                       | (((w >> 20) & 1) << 11) | (((w >> 21) & 0x3FF) << 1), 21)
+            wr(nxt)
+            nxt = self.add32(pc, u32(imm))
+        elif op == 0x67:
+            t = self.add32(a, u32(sext(w >> 20, 12))) & ~1
+            wr(nxt)
+            nxt = t
+        elif op == 0x63:
+            imm = sext((((w >> 31) & 1) << 12) | (((w >> 7) & 1) << 11)
+                       | (((w >> 25) & 0x3F) << 5) | (((w >> 8) & 0xF) << 1), 13)
+            eq = self.ucmp(a, b, "eq")
+            sl = self.scmp_lt(a, b)
+            ul = self.ucmp(a, b, "lessthan")
+            take = [eq, 1 - eq, 0, 0, sl, 1 - sl, ul, 1 - ul][f3]
+            if take:
+                nxt = self.add32(pc, u32(imm))
+        elif op == 0x03:
+            addr = self.add32(a, u32(sext(w >> 20, 12))) & 0xFFFF
+            width = [1, 2, 4, 0, 1, 2][f3]
+            wr(self._load(s, addr, width, f3 < 3))
+        elif op == 0x23:
+            imm = sext(((w >> 25) << 5) | ((w >> 7) & 31), 12)
+            addr = self.add32(a, u32(imm)) & 0xFFFF
+            self._store(s, addr, b, [1, 2, 4][f3])
+        elif op == 0x07:
+            addr = self.add32(a, u32(sext(w >> 20, 12))) & 0xFFFF
+            FR[rd] = self._load(s, addr, 4, False)
+        elif op == 0x27:
+            imm = sext(((w >> 25) << 5) | ((w >> 7) & 31), 12)
+            addr = self.add32(a, u32(imm)) & 0xFFFF
+            self._store(s, addr, FR[rs2], 4)
+        elif op == 0x53:
+            fa, fb = FR[rs1], FR[rs2]
+            if f7 in (0x00, 0x04, 0x08, 0x0C):
+                circ = {0x00: "add", 0x04: "add", 0x08: "mul", 0x0C: "div"}[f7]
+                if f7 == 0x04:
+                    fb ^= 0x80000000  # subtraction: flip the addend's sign
+                FR[rd] = self._fpu_word(circ, fa, fb)
+            elif f7 == 0x50:
+                out = self.fpu["cmp"].run(self._fpu_ext(fa, fb))
+                gate = {0: "le", 1: "lt", 2: "eq"}[f3]
+                wr(int(out[f"float32.cmp.{gate}.result"][0, 0].item()))
+            elif f7 == 0x10:  # FSGNJ family: result = {sign, rs1[30:0]}
+                sa = (FR[rs1] >> 31) & 1
+                sb = (FR[rs2] >> 31) & 1
+                if f3 == 0:
+                    s_out = sb                       # FSGNJ.S
+                elif f3 == 1:
+                    s_out = self._not1(sb)            # FSGNJN.S
+                else:
+                    s_out = self._bitwise("xor", sa << 31, sb << 31) >> 31  # FSGNJX.S
+                FR[rd] = (FR[rs1] & 0x7FFFFFFF) | (s_out << 31)
+            elif f7 == 0x60:  # FCVT.W.S: float -> signed int, gate-routed
+                wr(self.fcvt_w_s(FR[rs1]))
+            elif f7 == 0x68:  # FCVT.S.W: signed int -> float, gate-routed
+                FR[rd] = self.fcvt_s_w(R[rs1])
+            elif f7 == 0x78:
+                FR[rd] = a
+            elif f7 == 0x70:
+                wr(FR[rs1])
+        elif op == 0x0B:
+            wr(self.neur(a, b))
+        elif op == 0x73:
+            s["halted"] = True
+        elif op == 0x0F:
+            pass
+        elif op not in PAIRABLE:
+            raise ValueError(f"illegal instruction {w:#010x} at {pc:#06x}")
+        s["pc"] = nxt
+        return s
+
+    def _fpu_ext(self, fa, fb):
+        ext = {}
+        for i in range(32):
+            ext[f"$a[{i}]"] = (fa >> (31 - i)) & 1
+            ext[f"$b[{i}]"] = (fb >> (31 - i)) & 1
+        return ext
+
+    def _fpu_word(self, circ, fa, fb):
+        out = self.fpu[circ].run(self._fpu_ext(fa, fb))
+        p = f"float32.{circ}"
+        v = int(out[f"{p}.sign_out"][0, 0].item()) << 31
+        for k in range(8):
+            v |= int(out[f"{p}.exp_out.bit{k}"][0, 0].item()) << (23 + k)
+        for k in range(23):
+            v |= int(out[f"{p}.frac_out.bit{k}"][0, 0].item()) << k
+        return v
+
+    def run(self, state, max_cycles=400):
+        s = dict(state)
+        s["_mem_bits"] = torch.tensor(
+            [int_to_bits(byte, 8) for byte in s["mem"]], dtype=torch.float32)
+        n = 0
+        while not s["halted"] and n < max_cycles:
+            s = self.step(s)
+            n += 1
+        s.pop("_mem_bits", None)
+        return s, n
+
+# ============================================================================
+# neural_subleq8: threshold runtime, reference, assembler
+# ============================================================================
+SUBLEQ_MODEL = os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                     "variants", "neural_subleq8.safetensors")
+HALT_PC = 0xFF
+
+
+def load_tensors(path):
+    out = {}
+    with safe_open(path, framework="pt") as f:
+        for name in f.keys():
+            out[name] = f.get_tensor(name).float()
+    return out
+
+
+class SubleqThresholdCPU:
+    """SUBLEQ-8 runtime: memory traffic through the packed memory circuits,
+    the subtract/branch datapath through the subleq gates via the shipped
+    netlist. Instruction sequencing (three consecutive fetches) is wiring."""
+
+    def __init__(self, tensors, registry):
+        self.T = tensors
+        self.ne = NetlistEvaluator(tensors, registry, "subleq")
+
+    def _addr_decode(self, addr):
+        bits = torch.tensor([(addr >> (7 - i)) & 1 for i in range(8)],
+                            dtype=torch.float32)
+        w = self.T["memory.addr_decode.weight"]
+        b = self.T["memory.addr_decode.bias"]
+        return heaviside((w * bits).sum(dim=1) + b)
+
+    def mem_read(self, mem, addr):
+        sel = self._addr_decode(addr)
+        mem_bits = torch.tensor([[(byte >> (7 - i)) & 1 for i in range(8)]
+                                 for byte in mem], dtype=torch.float32)
+        and_w = self.T["memory.read.and.weight"]
+        and_b = self.T["memory.read.and.bias"]
+        or_w = self.T["memory.read.or.weight"]
+        or_b = self.T["memory.read.or.bias"]
+        out = 0
+        for bit in range(8):
+            inp = torch.stack([mem_bits[:, bit], sel], dim=1)
+            and_out = heaviside((inp * and_w[bit]).sum(dim=1) + and_b[bit])
+            out = (out << 1) | int(heaviside((and_out * or_w[bit]).sum() + or_b[bit]).item())
+        return out
+
+    def mem_write(self, mem, addr, value):
+        sel = self._addr_decode(addr)
+        data_bits = torch.tensor([(value >> (7 - i)) & 1 for i in range(8)],
+                                 dtype=torch.float32)
+        mem_bits = torch.tensor([[(byte >> (7 - i)) & 1 for i in range(8)]
+                                 for byte in mem], dtype=torch.float32)
+        sel_w = self.T["memory.write.sel.weight"]
+        sel_b = self.T["memory.write.sel.bias"]
+        nsel_w = self.T["memory.write.nsel.weight"].squeeze(1)
+        nsel_b = self.T["memory.write.nsel.bias"]
+        we = torch.ones_like(sel)
+        write_sel = heaviside((torch.stack([sel, we], dim=1) * sel_w).sum(dim=1) + sel_b)
+        nsel = heaviside(write_sel * nsel_w + nsel_b)
+        ao_w = self.T["memory.write.and_old.weight"]
+        ao_b = self.T["memory.write.and_old.bias"]
+        an_w = self.T["memory.write.and_new.weight"]
+        an_b = self.T["memory.write.and_new.bias"]
+        or_w = self.T["memory.write.or.weight"]
+        or_b = self.T["memory.write.or.bias"]
+        for bit in range(8):
+            old = mem_bits[:, bit]
+            dbit = data_bits[bit].expand(256)
+            a_old = heaviside((torch.stack([old, nsel], dim=1) * ao_w[:, bit]).sum(dim=1) + ao_b[:, bit])
+            a_new = heaviside((torch.stack([dbit, write_sel], dim=1) * an_w[:, bit]).sum(dim=1) + an_b[:, bit])
+            mem_bits[:, bit] = heaviside(
+                (torch.stack([a_old, a_new], dim=1) * or_w[:, bit]).sum(dim=1) + or_b[:, bit])
+        return [int(sum(int(mem_bits[i, b].item()) << (7 - b) for b in range(8)))
+                for i in range(256)]
+
+    def step(self, state):
+        if state["halted"]:
+            return state
+        s = dict(state)
+        s["mem"] = state["mem"][:]
+        pc = s["pc"]
+        A = self.mem_read(s["mem"], pc)
+        B = self.mem_read(s["mem"], (pc + 1) & 0xFF)
+        C = self.mem_read(s["mem"], (pc + 2) & 0xFF)
+        a = self.mem_read(s["mem"], A)
+        b = self.mem_read(s["mem"], B)
+        ext = {}
+        for k in range(8):
+            ext[f"$a[{k}]"] = (a >> k) & 1
+            ext[f"$b[{k}]"] = (b >> k) & 1
+            ext[f"$pc[{k}]"] = (pc >> k) & 1
+            ext[f"$c[{k}]"] = (C >> k) & 1
+        out = self.ne.run(ext)
+        r = sum(int(out[f"subleq.sub.fa{k}.ha2.sum.layer2"][0, 0].item()) << k
+                for k in range(8))
+        next_pc = sum(int(out[f"subleq.pc_mux.bit{k}.or"][0, 0].item()) << k
+                      for k in range(8))
+        s["mem"] = self.mem_write(s["mem"], B, r)
+        s["pc"] = next_pc
+        if next_pc == HALT_PC:
+            s["halted"] = True
+        return s
+
+    def run(self, state, max_cycles=200):
+        s = state
+        n = 0
+        while not s["halted"] and n < max_cycles:
+            s = self.step(s)
+            n += 1
+        return s, n
+
+
+def ref_step(state):
+    if state["halted"]:
+        return state
+    s = dict(state)
+    s["mem"] = state["mem"][:]
+    pc = s["pc"]
+    A, B, C = s["mem"][pc], s["mem"][(pc + 1) & 0xFF], s["mem"][(pc + 2) & 0xFF]
+    r = (s["mem"][B] - s["mem"][A]) & 0xFF
+    s["mem"][B] = r
+    s["pc"] = C if (r == 0 or r >= 0x80) else (pc + 3) & 0xFF
+    if s["pc"] == HALT_PC:
+        s["halted"] = True
+    return s
+
+
+def sq(*triples):
+    """Assemble SUBLEQ triples into a 256-byte image."""
+    mem = [0] * 256
+    i = 0
+    for t in triples:
+        for v in t:
+            mem[i] = v & 0xFF
+            i += 1
+    return mem
+
+
+def subleq_test() -> int:
+    T = load_tensors(SUBLEQ_MODEL)
+    reg = load_metadata(SUBLEQ_MODEL)["signal_registry"]
+
+    # 1. Exhaustive datapath: all 65,536 (a, b) pairs through the shipped
+    # wiring in one batched netlist evaluation.
+    ne = NetlistEvaluator(T, reg, "subleq")
+    a_v = torch.arange(65536, dtype=torch.long) % 256
+    b_v = torch.arange(65536, dtype=torch.long) // 256
+    ext = {}
+    for k in range(8):
+        ext[f"$a[{k}]"] = ((a_v >> k) & 1).float()
+        ext[f"$b[{k}]"] = ((b_v >> k) & 1).float()
+        ext[f"$pc[{k}]"] = torch.zeros(65536)
+        ext[f"$c[{k}]"] = torch.zeros(65536)
+    out = ne.run(ext)
+    r = torch.zeros(65536, dtype=torch.float64)
+    for k in range(8):
+        r += out[f"subleq.sub.fa{k}.ha2.sum.layer2"][:, 0].double() * (1 << k)
+    exp_r = ((b_v - a_v) & 0xFF).double()
+    leq = out["subleq.leq"][:, 0].double()
+    exp_leq = ((exp_r == 0) | (exp_r >= 128)).double()
+    ok_r = int((r == exp_r).sum().item())
+    ok_l = int((leq == exp_leq).sum().item())
+    print(f"datapath sub  exhaustive: {ok_r}/65536")
+    print(f"datapath leq  exhaustive: {ok_l}/65536")
+    if ok_r != 65536 or ok_l != 65536:
+        return 1
+
+    # 2. PC mux + increment across representative PC/target/leq settings.
+    pcs = torch.tensor([0, 3, 5, 100, 250, 252], dtype=torch.long).repeat_interleave(2)
+    cs = torch.tensor([7, 7, 42, 42, 255, 255, 0, 0, 9, 9, 30, 30], dtype=torch.long)
+    a2 = torch.tensor([0, 1] * 6, dtype=torch.long)   # a=0,b=0 -> leq; a=1,b=0 -> r=255 leq... use b to vary
+    b2 = torch.tensor([0, 1] * 6, dtype=torch.long)   # (0,0)->r0 leq taken; (1,1)->r0 leq taken
+    # craft: even rows leq taken (a=b), odd rows not taken (b-a=1 -> r=1 >0)
+    a2 = torch.tensor([5, 4] * 6, dtype=torch.long)
+    b2 = torch.tensor([5, 5] * 6, dtype=torch.long)
+    ext = {}
+    for k in range(8):
+        ext[f"$a[{k}]"] = ((a2 >> k) & 1).float()
+        ext[f"$b[{k}]"] = ((b2 >> k) & 1).float()
+        ext[f"$pc[{k}]"] = ((pcs >> k) & 1).float()
+        ext[f"$c[{k}]"] = ((cs >> k) & 1).float()
+    out = ne.run(ext)
+    npc = torch.zeros(12, dtype=torch.float64)
+    for k in range(8):
+        npc += out[f"subleq.pc_mux.bit{k}.or"][:, 0].double() * (1 << k)
+    exp_npc = torch.where(torch.arange(12) % 2 == 0, cs.double(), ((pcs + 3) & 0xFF).double())
+    ok_pc = int((npc == exp_npc).sum().item())
+    print(f"datapath pc mux/inc     : {ok_pc}/12")
+    if ok_pc != 12:
+        return 1
+
+    # 3. Program suite, threshold vs reference, full-state lockstep.
+    cpu = SubleqThresholdCPU(T, reg)
+    Z = 0x80  # scratch zero cell
+
+    progs = []
+    # negate-copy: M[dst] = -M[src] (dst pre-zeroed): subleq src, dst; halt.
+    m = sq((0x90, 0x91, 3), (Z, Z, HALT_PC))
+    m[0x90], m[0x91] = 42, 0
+    progs.append(("neg_copy", m, 30, lambda mm: mm[0x91] == ((-42) & 0xFF)))
+    # add: M[d] += M[s] via double negation through Z.
+    m = sq((0x90, Z, 3), (Z, 0x91, 6), (Z, Z, 9), (Z, Z, HALT_PC))
+    m[0x90], m[0x91] = 17, 25
+    progs.append(("add", m, 30, lambda mm: mm[0x91] == 42))
+    # countdown loop: decrement M[n] by M[one] until <= 0, counting into 0x92
+    # via add idiom is long; simplest loop: subleq one, n, done at 0.
+    m = sq((0x93, 0x90, 6), (Z, Z, 0), (Z, Z, HALT_PC))
+    m[0x90], m[0x93] = 5, 1
+    progs.append(("countdown", m, 60, lambda mm: mm[0x90] == 0))
+    # halt-immediately
+    m = sq((Z, Z, HALT_PC))
+    progs.append(("halt", m, 5, lambda mm: True))
+
+    all_ok = True
+    for name, mem, budget, check in progs:
+        ts = {"pc": 0, "mem": mem[:], "halted": False}
+        rs = {"pc": 0, "mem": mem[:], "halted": False}
+        cyc = 0
+        ok = True
+        while not ts["halted"] and cyc < budget:
+            ts = cpu.step(ts)
+            rs = ref_step(rs)
+            cyc += 1
+            if ts["pc"] != rs["pc"] or ts["mem"] != rs["mem"]:
+                ok = False
+                break
+        ok = ok and ts["halted"] and rs["halted"] and check(ts["mem"])
+        all_ok &= ok
+        print(f"  {name:12} {'PASS' if ok else 'FAIL'}  ({cyc} cyc, lockstep)")
+
+    print("SUBLEQ:", "PASS" if all_ok else "FAIL")
+    return 0 if all_ok else 1
+
+# ============================================================================
+# neural_rv32 test suites
+# ============================================================================
+F1_0 = struct.unpack("<I", struct.pack("<f", 1.5))[0]
+F2_5 = struct.unpack("<I", struct.pack("<f", 2.5))[0]
+F10 = struct.unpack("<I", struct.pack("<f", 10.0))[0]
+
+
+def lockstep(cpu, mem, budget, name, check=None):
+    ts = rv_state(mem)
+    rs = rv_state(mem)
+    import torch
+    from eval_all import int_to_bits
+    ts["_mem_bits"] = torch.tensor([int_to_bits(b, 8) for b in ts["mem"]],
+                                   dtype=torch.float32)
+    cyc = 0
+    ok = True
+    while not ts["halted"] and cyc < budget:
+        ts = cpu.step(ts)
+        # reference is strictly single-issue; advance it until PCs realign
+        rs = RefRV32.step(rs)
+        if rs["pc"] != ts["pc"] and not rs["halted"]:
+            rs = RefRV32.step(rs)
+        cyc += 1
+        if (ts["pc"] != rs["pc"] or ts["regs"] != rs["regs"]
+                or ts["fregs"] != rs["fregs"] or ts["halted"] != rs["halted"]):
+            ok = False
+            break
+    if ok:
+        ok = ts["halted"] and ts["mem"] == rs["mem"] and ts["console"] == rs["console"]
+    if ok and check is not None:
+        ok = check(ts)
+    print(f"  {name:22} {'PASS' if ok else 'FAIL'}  ({cyc} cyc)")
+    if not ok:
+        print(f"    pc t={ts['pc']:#x} r={rs['pc']:#x} halted t={ts['halted']} r={rs['halted']}")
+        for i in range(32):
+            if ts["regs"][i] != rs["regs"][i]:
+                print(f"    x{i}: t={ts['regs'][i]:#x} r={rs['regs'][i]:#x}")
+    return ok
+
+
+def prog_int_alu():
+    a = Asm()
+    a.addi(1, 0, 100)          # x1 = 100
+    a.addi(2, 0, -7)           # x2 = -7
+    a.add(3, 1, 2)             # 93
+    a.sub(4, 1, 2)             # 107
+    a.xor(5, 1, 2)
+    a.or_(6, 1, 2)
+    a.and_(7, 1, 2)
+    a.slt(8, 2, 1)             # 1 (signed)
+    a.sltu(9, 2, 1)            # 0 (unsigned: -7 is huge)
+    a.slli(10, 1, 3)           # 800
+    a.srli(11, 2, 4)
+    a.srai(12, 2, 4)           # -1
+    a.slti(13, 2, 0)           # 1
+    a.sltiu(14, 1, 200)        # 1
+    a.xori(15, 1, 0xFF)
+    a.ori(16, 1, 0x0F)
+    a.andi(17, 1, 0x3C)
+    a.lui(18, 0x12345)
+    a.auipc(19, 1)
+    a.ecall()
+    return a.assemble(), 40, lambda s: s["regs"][3] == 93 and s["regs"][12] == u32(-1)
+
+
+def prog_branches():
+    a = Asm()
+    a.addi(1, 0, 5)
+    a.addi(2, 0, -3)
+    a.addi(10, 0, 0)
+    a.blt(2, 1, "l1")          # signed taken
+    a.addi(10, 10, 100)
+    a.label("l1")
+    a.bltu(1, 2, "l2")         # unsigned taken (5 < 0xFFFF..FD)
+    a.addi(10, 10, 200)
+    a.label("l2")
+    a.beq(1, 1, "l3")
+    a.addi(10, 10, 400)
+    a.label("l3")
+    a.bge(1, 2, "l4")          # taken
+    a.addi(10, 10, 800)
+    a.label("l4")
+    a.bne(1, 2, "done")
+    a.addi(10, 10, 1600)
+    a.label("done")
+    a.ecall()
+    return a.assemble(), 30, lambda s: s["regs"][10] == 0
+
+
+def prog_fib():
+    a = Asm()
+    a.addi(1, 0, 0)            # a
+    a.addi(2, 0, 1)            # b
+    a.addi(3, 0, 10)           # n
+    a.label("loop")
+    a.beq(3, 0, "done")
+    a.add(4, 1, 2)
+    a.add(1, 0, 2)             # a = b  (independent pair candidates)
+    a.add(2, 0, 4)             # b = t
+    a.addi(3, 3, -1)
+    a.jal(0, "loop")
+    a.label("done")
+    a.ecall()
+    return a.assemble(), 200, lambda s: s["regs"][1] == 55
+
+
+def prog_mem_call():
+    a = Asm()
+    a.addi(2, 0, 0x800)        # sp
+    a.addi(1, 0, 0x77)
+    a.sb(1, 0, 0x400)
+    a.lb(3, 0, 0x400)
+    a.lbu(4, 0, 0x400)
+    a.addi(1, 0, -2)
+    a.sh(1, 0, 0x404)
+    a.lh(5, 0, 0x404)          # sign-extended
+    a.lhu(6, 0, 0x404)
+    a.jal(1, "fn")             # call
+    a.ecall()
+    a.label("fn")
+    a.addi(2, 2, -4)
+    a.sw(1, 2, 0)              # push ra
+    a.lw(7, 2, 0)
+    a.addi(2, 2, 4)
+    a.jalr(0, 1, 0)            # ret
+    return a.assemble(), 40, lambda s: s["regs"][5] == u32(-2) and s["regs"][6] == 0xFFFE
+
+
+def prog_muldiv():
+    a = Asm()
+    a.addi(1, 0, -50)
+    a.addi(2, 0, 7)
+    a.mul(3, 1, 2)             # -350
+    a.mulh(4, 1, 2)            # high of signed product = -1
+    a.mulhu(5, 1, 2)
+    a.div(6, 1, 2)             # -7
+    a.rem(7, 1, 2)             # -1
+    a.divu(8, 1, 2)
+    a.remu(9, 1, 2)
+    a.addi(10, 0, 0)
+    a.div(11, 1, 10)           # div by zero -> -1
+    a.rem(12, 1, 10)           # rem by zero -> dividend
+    a.ecall()
+    return a.assemble(), 30, lambda s: (s["regs"][3] == u32(-350)
+                                        and s["regs"][6] == u32(-7)
+                                        and s["regs"][7] == u32(-1)
+                                        and s["regs"][11] == 0xFFFFFFFF)
+
+
+def prog_float():
+    a = Asm()
+    a.lui(1, 0x1)              # x1 = 0x1000 data base
+    a.flw(1, 1, 0)             # f1 = 1.5
+    a.flw(2, 1, 4)             # f2 = 2.5
+    a.fadd_s(3, 1, 2)          # 4.0
+    a.fmul_s(4, 1, 2)          # 3.75
+    a.fdiv_s(5, 2, 1)          # 2.5 / 1.5
+    a.fsub_s(6, 2, 1)          # 1.0
+    a.fsw(3, 1, 8)
+    a.fle_s(10, 1, 2)          # 1
+    a.flt_s(11, 2, 1)          # 0
+    a.feq_s(12, 1, 1)          # 1
+    a.fmv_x_w(13, 3)
+    a.ecall()
+    mem = a.assemble()
+    for i, wv in ((0, F1_0), (4, F2_5)):
+        for b in range(4):
+            mem[0x1000 + i + b] = (wv >> (8 * b)) & 0xFF
+    f4 = struct.unpack("<I", struct.pack("<f", 4.0))[0]
+    return mem, 30, lambda s: (s["regs"][10] == 1 and s["regs"][11] == 0
+                               and s["regs"][12] == 1 and s["regs"][13] == f4)
+
+
+def prog_neur():
+    # XOR of two bits computed by a two-layer network of NEUR instructions:
+    # h1 = OR(a,b) = H(a + b - 1); h2 = NAND(a,b) = H(-a - b + 1);
+    # y = AND(h1,h2) = H(h1 + h2 - 2).
+    def const(a, reg, v):
+        a.lui(reg, v >> 12)
+        a.ori(reg, reg, v & 0x7FF)
+
+    a = Asm()
+    a.addi(1, 0, 0b10)                     # inputs: bit1=a=1, bit0=b=0
+    const(a, 2, (0x1F << 16) | 0x03)       # OR: pos=both, bias=-1 (0x1F sext5)
+    a.neur(3, 1, 2)
+    const(a, 4, (0x01 << 16) | (0x03 << 8))  # NAND: neg=both, bias=+1
+    a.neur(5, 1, 4)
+    a.slli(6, 3, 1)
+    a.or_(6, 6, 5)                          # pack h1,h2 into bits 1,0
+    const(a, 7, (0x1E << 16) | 0x03)        # AND: pos=both, bias=-2 (0x1E sext5)
+    a.neur(8, 6, 7)                         # y = XOR(a,b) = 1
+    a.ecall()
+    return a.assemble(), 20, lambda s: s["regs"][8] == 1 and s["regs"][3] == 1 and s["regs"][5] == 1
+
+
+def prog_mmio():
+    a = Asm()
+    a.lui(1, 0x10)             # 0x10000 > 0xFFFF? keep in range: use addi chain
+    a.addi(1, 0, 0x7F8)
+    a.slli(1, 1, 5)            # 0xFF00
+    a.addi(2, 0, ord("H"))
+    a.sb(2, 1, 0)
+    a.addi(2, 0, ord("I"))
+    a.sb(2, 1, 0)
+    a.ecall()
+    return a.assemble(), 20, lambda s: s["console"] == "HI"
+
+
+def prog_fcvt():
+    # int -> float, add 1.0, float -> int: (7 + 1.0) truncated = 8; and a
+    # round-trip of -5 through float and back = -5.
+    import struct as _st
+    a = Asm()
+    a.addi(1, 0, 7)
+    a.fcvt_s_w(0, 1)          # f0 = 7.0
+    a.lui(2, 0x1); a.flw(1, 2, 0)   # f1 = 1.0 from data
+    a.fadd_s(2, 0, 1)        # f2 = 8.0
+    a.fcvt_w_s(3, 2)         # x3 = 8
+    a.addi(4, 0, -5)
+    a.fcvt_s_w(4, 4)         # f4 = -5.0
+    a.fcvt_w_s(5, 4)         # x5 = -5
+    a.ecall()
+    mem = a.assemble()
+    one = _st.unpack("<I", _st.pack("<f", 1.0))[0]
+    for b in range(4):
+        mem[0x1000 + b] = (one >> (8 * b)) & 0xFF
+    return mem, 30, lambda s: s["regs"][3] == 8 and s["regs"][5] == (-5 & 0xFFFFFFFF)
+
+
+def rv32_netlist_check() -> bool:
+    """Verify the rv32-specific sub-circuits (NEUR, hazard, MULHU accumulator)
+    are recoverable and correct from the shipped .inputs metadata alone."""
+    import random
+    T = {}
+    with safe_open(RV32_MODEL, framework="pt") as f:
+        for name in f.keys():
+            T[name] = f.get_tensor(name).float()
+    reg = load_metadata(RV32_MODEL)["signal_registry"]
+    rng = random.Random(0)
+    ok = True
+
+    ne = NetlistEvaluator(T, reg, "rv32.neur")
+    N = 512
+    xs = [rng.getrandbits(8) for _ in range(N)]
+    ps = [rng.getrandbits(8) for _ in range(N)]
+    ns = [rng.getrandbits(8) for _ in range(N)]
+    bs = [rng.randint(-16, 15) for _ in range(N)]
+    ext = {}
+    for i in range(8):
+        ext[f"$rv32_neur_x[{i}]"] = torch.tensor([(x >> (7 - i)) & 1 for x in xs]).float()
+        ext[f"$rv32_neur_pos[{i}]"] = torch.tensor([(p >> (7 - i)) & 1 for p in ps]).float()
+        ext[f"$rv32_neur_neg[{i}]"] = torch.tensor([(n >> (7 - i)) & 1 for n in ns]).float()
+    for k in range(6):
+        ext[f"$rv32_neur_bias[{k}]"] = torch.tensor(
+            [((b & 0x1F) >> k) & 1 if k < 5 else (b >> 4) & 1 for b in bs]).float()
+    got = ne.run(ext)["rv32.neur.out"][:, 0]
+    exp = torch.tensor([1.0 if (bin(x & p).count("1") - bin(x & n).count("1") + b) >= 0
+                        else 0.0 for x, p, n, b in zip(xs, ps, ns, bs)])
+    ok &= bool((got == exp).all())
+
+    ne = NetlistEvaluator(T, reg, "rv32.hazard.rd1_rs1b")
+    ii = [rng.getrandbits(5) for _ in range(N)]
+    jj = [rng.getrandbits(5) for _ in range(N)]
+    ext = {}
+    for k in range(5):
+        ext[f"$rv32_hz_rd1_rs1b_i[{k}]"] = torch.tensor([(i >> k) & 1 for i in ii]).float()
+        ext[f"$rv32_hz_rd1_rs1b_j[{k}]"] = torch.tensor([(j >> k) & 1 for j in jj]).float()
+    got = ne.run(ext)["rv32.hazard.rd1_rs1b.all"][:, 0]
+    exp = torch.tensor([1.0 if i == j else 0.0 for i, j in zip(ii, jj)])
+    ok &= bool((got == exp).all())
+
+    ne = NetlistEvaluator(T, reg, "rv32.mulacc")
+    A = [rng.getrandbits(32) for _ in range(256)]
+    B = [rng.getrandbits(32) for _ in range(256)]
+    ext = {}
+    for I in range(32):
+        for J in range(32):
+            ext[f"alu.alu32bit.mul.pp.a{I}b{J}"] = torch.tensor(
+                [((a >> (31 - I)) & 1) & ((b >> (31 - J)) & 1) for a, b in zip(A, B)]).float()
+    out = ne.run(ext)
+    got = torch.zeros(len(A), dtype=torch.float64)
+    for k in range(32):
+        got += out[f"rv32.mulacc.s30.fa{32 + k}.ha2.sum.layer2"][:, 0].double() * (2.0 ** k)
+    exp = torch.tensor([float(((a * b) >> 32) & 0xFFFFFFFF) for a, b in zip(A, B)],
+                       dtype=torch.float64)
+    ok &= bool((got == exp).all())
+
+    print(f"  rv32 netlist self-check (NEUR/hazard/MULHU from metadata)  "
+          f"{'PASS' if ok else 'FAIL'}")
+    return ok
+
+
+def rv32_test() -> int:
+    print("Loading neural_rv32...")
+    cpu = Rv32ThresholdCPU()
+    suite = [
+        ("int_alu", prog_int_alu),
+        ("branches", prog_branches),
+        ("fib", prog_fib),
+        ("mem_call", prog_mem_call),
+        ("muldiv", prog_muldiv),
+        ("float_subset", prog_float),
+        ("fcvt_roundtrip", prog_fcvt),
+        ("neur_xor_net", prog_neur),
+        ("mmio_print", prog_mmio),
+    ]
+    all_ok = True
+    for name, builder in suite:
+        mem, budget, check = builder()
+        all_ok &= lockstep(cpu, mem, budget, name, check)
+    all_ok &= rv32_netlist_check()
+    print(f"dual-issue pairs retired: {cpu.pairs_issued}")
+    print("RV32:", "PASS" if all_ok else "FAIL")
+    return 0 if all_ok else 1
+
+PROG = r"""
+__attribute__((naked)) void _start(void){
+  __asm__ volatile("li sp, 0xF000\n call main\n ecall\n");
+}
+static int gcd(int a,int b){ while(b){ int t=a%b; a=b; b=t;} return a; }
+int fib(int n){ int a=0,b=1; for(int i=0;i<n;i++){int t=a+b;a=b;b=t;} return a; }
+int main(void){
+  int arr[8]={5,2,9,1,7,3,8,4};
+  for(int i=0;i<8;i++) for(int j=i+1;j<8;j++)
+    if(arr[j]<arr[i]){int t=arr[i];arr[i]=arr[j];arr[j]=t;}
+  int s=0; for(int i=0;i<8;i++) s+=arr[i]*(i+1);
+  return gcd(48,36) + fib(10) + s + (arr[0]==1?1000:0);
+}
+"""
+EXPECTED = 1292   # gcd(48,36)=12, fib(10)=55, weighted sorted sum=225... verified natively
+
+
+def have_riscv_clang() -> bool:
+    if not shutil.which("clang"):
+        return False
+    with tempfile.TemporaryDirectory() as d:
+        c = os.path.join(d, "t.c")
+        o = os.path.join(d, "t.o")
+        open(c, "w").write("int f(void){return 1;}")
+        r = subprocess.run(
+            ["clang", "-target", "riscv32-unknown-elf", "-march=rv32im",
+             "-mabi=ilp32", "-c", c, "-o", o],
+            capture_output=True)
+        return r.returncode == 0
+
+
+def rv32c_test() -> int:
+    if not have_riscv_clang():
+        print("SKIP: no riscv32 clang target available")
+        return 0
+
+    with tempfile.TemporaryDirectory() as d:
+        c = os.path.join(d, "prog.c")
+        o = os.path.join(d, "prog.o")
+        open(c, "w").write(PROG)
+        subprocess.run(
+            ["clang", "-target", "riscv32-unknown-elf", "-march=rv32im",
+             "-mabi=ilp32", "-O1", "-nostdlib", "-ffreestanding",
+             "-c", c, "-o", o], check=True, capture_output=True)
+        obj = open(o, "rb").read()
+
+    mem, entry = load(obj, base=0, mem_size=65536)
+    cpu = Rv32ThresholdCPU()
+    st = rv_state(mem)
+    st["pc"] = entry
+    fin, cyc = cpu.run(st, max_cycles=5000)
+    got = fin["regs"][10]
+    ok = fin["halted"] and got == EXPECTED
+    print(f"compiled C (gcd+fib+sort, rv32im): a0={got} "
+          f"expected {EXPECTED}  {cyc} instrs  {'PASS' if ok else 'FAIL'}")
+    return 0 if ok else 1
+
+# ============================================================================
+# Entry point
+# ============================================================================
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description="neural_subleq8 / neural_rv32 test suites")
+    ap.add_argument("which", nargs="?", default="all",
+                    choices=["all", "subleq", "rv32", "rv32-c"])
+    args = ap.parse_args()
+    rc = 0
+    if args.which in ("all", "subleq"):
+        rc |= subleq_test()
+    if args.which in ("all", "rv32"):
+        rc |= rv32_test()
+    if args.which in ("all", "rv32-c"):
+        rc |= rv32c_test()
+    return rc
+
+
+if __name__ == "__main__":
+    sys.exit(main())

neural_computer.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb4725ab213771fa0f20c42d13a43b913cb33f1762c6a8c4b4670856f8d3592c
+size 32717681

play.py

@@ -0,0 +1,262 @@
+"""
+Hands-on playground for the 8bit-threshold-computer.
+
+Loads a safetensors model, reads its manifest, and exercises threshold
+circuits at every level: raw Boolean gates, 8-bit ALU arithmetic and
+comparators, multi-layer modular arithmetic, and a manifest-sized CPU
+runtime running a small assembled program end-to-end through the
+threshold weights.
+
+The CPU demo defaults to the small (1 KB) profile so the run finishes in
+a fraction of a second. Larger profiles (4 KB, 64 KB) take proportionally
+longer because every memory access decodes against every address line.
+
+Usage:
+    python play.py                                  # fast 1KB demo
+    python play.py --model neural_computer.safetensors   # full 64KB
+    python play.py --model variants/neural_alu8.safetensors --skip-cpu  # ALU only
+"""
+
+from __future__ import annotations
+import argparse
+import os
+import sys
+
+import torch
+from safetensors import safe_open
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+# Reuse the variant-aware CPU runtime from eval_all.py
+from eval_all import GenericThresholdCPU, builtin_program
+
+
+def heaviside(x):
+    return (x >= 0).float()
+
+
+def load_tensors(path):
+    out = {}
+    with safe_open(path, framework="pt") as f:
+        for name in f.keys():
+            out[name] = f.get_tensor(name).float()
+    return out
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Threshold computer playground")
+    parser.add_argument(
+        "--model", type=str,
+        default=os.path.join(os.path.dirname(__file__),
+                             "variants", "neural_computer8_small.safetensors"),
+        help="Path to a .safetensors variant"
+    )
+    parser.add_argument("--skip-cpu", action="store_true",
+                        help="Skip the CPU program demo (useful for pure-ALU files)")
+    args = parser.parse_args()
+
+    print("Loading", args.model)
+    T = load_tensors(args.model)
+
+    DATA_BITS = int(T["manifest.data_bits"].item())
+    ADDR_BITS = int(T["manifest.addr_bits"].item())
+    MEM_BYTES = int(T["manifest.memory_bytes"].item())
+    REGISTERS = int(T["manifest.registers"].item())
+    print(f"Manifest: data={DATA_BITS}-bit, addr={ADDR_BITS}-bit, mem={MEM_BYTES}B, regs={REGISTERS}")
+    print(f"Tensors: {len(T):,}")
+    print(f"Total params: {sum(t.numel() for t in T.values()):,}")
+    print()
+
+    def gate(name, inputs):
+        w = T[name + ".weight"].view(-1)
+        b = T[name + ".bias"].view(-1)
+        return int(heaviside((torch.tensor(inputs, dtype=torch.float32) * w).sum() + b).item())
+
+    def xor(prefix, inputs):
+        a, b_ = inputs
+        h_or = gate(f"{prefix}.layer1.or", [a, b_])
+        h_nand = gate(f"{prefix}.layer1.nand", [a, b_])
+        return gate(f"{prefix}.layer2", [h_or, h_nand])
+
+    def xor_neuron(prefix, inputs):
+        a, b_ = inputs
+        h1 = gate(f"{prefix}.layer1.neuron1", [a, b_])
+        h2 = gate(f"{prefix}.layer1.neuron2", [a, b_])
+        return gate(f"{prefix}.layer2", [h1, h2])
+
+    def int_to_bits_msb(v, n):
+        return [(v >> (n - 1 - i)) & 1 for i in range(n)]
+
+    def bits_msb_to_int(bits):
+        out = 0
+        for b in bits:
+            out = (out << 1) | int(b)
+        return out
+
+    # ---------- Demo 1: Boolean gates ----------
+    print("=" * 64)
+    print(" Demo 1: Boolean threshold gates")
+    print("=" * 64)
+    truth_2 = [(0, 0), (0, 1), (1, 0), (1, 1)]
+    for gname in ["and", "or", "nand", "nor", "implies"]:
+        row = " ".join(f"{a}{b}->{gate(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
+        print(f"  {gname:8} {row}")
+    for gname in ["xor", "xnor", "biimplies"]:
+        row = " ".join(f"{a}{b}->{xor_neuron(f'boolean.{gname}', [a, b])}" for a, b in truth_2)
+        print(f"  {gname:8} {row}")
+    print(f"  not      0->{gate('boolean.not', [0])} 1->{gate('boolean.not', [1])}")
+    print()
+
+    # ---------- Demo 2: 8-bit ALU arithmetic ----------
+    print("=" * 64)
+    print(" Demo 2: 8-bit ALU arithmetic (every gate is threshold logic)")
+    print("=" * 64)
+
+    def fa(prefix, a, b, cin):
+        s1 = xor(f"{prefix}.ha1.sum", [a, b])
+        c1 = gate(f"{prefix}.ha1.carry", [a, b])
+        s2 = xor(f"{prefix}.ha2.sum", [s1, cin])
+        c2 = gate(f"{prefix}.ha2.carry", [s1, cin])
+        return s2, gate(f"{prefix}.carry_or", [c1, c2])
+
+    def alu_add(a, b):
+        a_lsb = list(reversed(int_to_bits_msb(a, 8)))
+        b_lsb = list(reversed(int_to_bits_msb(b, 8)))
+        carry = 0
+        sum_lsb = []
+        for i in range(8):
+            s, carry = fa(f"arithmetic.ripplecarry8bit.fa{i}", a_lsb[i], b_lsb[i], carry)
+            sum_lsb.append(s)
+        return bits_msb_to_int(list(reversed(sum_lsb))), carry
+
+    def alu_sub(a, b):
+        a_lsb = list(reversed(int_to_bits_msb(a, 8)))
+        b_lsb = list(reversed(int_to_bits_msb(b, 8)))
+        carry = 1
+        diff_lsb = []
+        for i in range(8):
+            notb = gate(f"arithmetic.sub8bit.notb{i}", [b_lsb[i]])
+            x1 = xor(f"arithmetic.sub8bit.fa{i}.xor1", [a_lsb[i], notb])
+            x2 = xor(f"arithmetic.sub8bit.fa{i}.xor2", [x1, carry])
+            and1 = gate(f"arithmetic.sub8bit.fa{i}.and1", [a_lsb[i], notb])
+            and2 = gate(f"arithmetic.sub8bit.fa{i}.and2", [x1, carry])
+            carry = gate(f"arithmetic.sub8bit.fa{i}.or_carry", [and1, and2])
+            diff_lsb.append(x2)
+        return bits_msb_to_int(list(reversed(diff_lsb))), carry
+
+    def alu_compare(a, b, kind):
+        # Walks the bit-cascade comparator family: per-bit gt/lt/eq, cascaded
+        # eq_prefix, cascade.gt/lt, and the final OR/AND gates. Bit 0 is MSB.
+        a_msb = int_to_bits_msb(a, 8)
+        b_msb = int_to_bits_msb(b, 8)
+        bit_gt = [gate(f"arithmetic.cmp8bit.bit{i}.gt", [a_msb[i], b_msb[i]]) for i in range(8)]
+        bit_lt = [gate(f"arithmetic.cmp8bit.bit{i}.lt", [a_msb[i], b_msb[i]]) for i in range(8)]
+        bit_eq = []
+        for i in range(8):
+            eq_and = gate(f"arithmetic.cmp8bit.bit{i}.eq.layer1.and", [a_msb[i], b_msb[i]])
+            eq_nor = gate(f"arithmetic.cmp8bit.bit{i}.eq.layer1.nor", [a_msb[i], b_msb[i]])
+            bit_eq.append(gate(f"arithmetic.cmp8bit.bit{i}.eq", [eq_and, eq_nor]))
+        cas_gt = [bit_gt[0]]
+        cas_lt = [bit_lt[0]]
+        for i in range(1, 8):
+            eq_pref = gate(f"arithmetic.cmp8bit.cascade.eq_prefix.bit{i}", bit_eq[:i])
+            cas_gt.append(gate(f"arithmetic.cmp8bit.cascade.gt.bit{i}", [eq_pref, bit_gt[i]]))
+            cas_lt.append(gate(f"arithmetic.cmp8bit.cascade.lt.bit{i}", [eq_pref, bit_lt[i]]))
+        if kind == "greaterthan":
+            return gate("arithmetic.greaterthan8bit", cas_gt)
+        if kind == "lessthan":
+            return gate("arithmetic.lessthan8bit", cas_lt)
+        if kind == "eq":
+            return gate("arithmetic.equality8bit", bit_eq)
+        raise ValueError(kind)
+
+    def alu_mul(a, b):
+        a_bits = int_to_bits_msb(a, 8)
+        b_bits = int_to_bits_msb(b, 8)
+        result = 0
+        for j in range(8):
+            if b_bits[j] == 0:
+                continue
+            row = 0
+            for i in range(8):
+                pp = gate(f"alu.alu8bit.mul.pp.a{i}b{j}", [a_bits[i], b_bits[j]])
+                row |= (pp << (7 - i))
+            shift = 7 - j
+            result, _ = alu_add(result & 0xFF, (row << shift) & 0xFF)
+        return result & 0xFF
+
+    cases_arith = [(5, 3), (37, 100), (200, 99), (255, 1), (127, 128), (15, 17)]
+    print("ADD:")
+    for a, b in cases_arith:
+        r, c = alu_add(a, b)
+        e = (a + b) & 0xFF
+        print(f"  {a:3} + {b:3} = {r:3} (carry={c})  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print("SUB:")
+    for a, b in cases_arith:
+        r, c = alu_sub(a, b)
+        e = (a - b) & 0xFF
+        print(f"  {a:3} - {b:3} = {r:3} (no_borrow={c})  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print("CMP:")
+    for a, b in [(50, 30), (30, 50), (77, 77), (255, 0), (0, 255), (128, 127)]:
+        gt = alu_compare(a, b, "greaterthan")
+        lt = alu_compare(a, b, "lessthan")
+        eq = alu_compare(a, b, "eq")
+        print(f"  {a:3} vs {b:3} -> GT={gt} LT={lt} EQ={eq}")
+    print("MUL (low 8 bits):")
+    for a, b in [(12, 11), (15, 17), (8, 32), (200, 3), (0, 99), (1, 255)]:
+        r = alu_mul(a, b)
+        e = (a * b) & 0xFF
+        print(f"  {a:3} * {b:3} = {r:3}  expected {e:3}  [{'OK' if r == e else 'FAIL'}]")
+    print()
+
+    # ---------- Demo 3: mod-5 divisibility ----------
+    print("=" * 64)
+    print(" Demo 3: mod-5 divisibility (multi-layer, hand-constructed)")
+    print("=" * 64)
+
+    def mod5(v):
+        # Per-multiple-of-5 match (k0, k5, ..., k255): each k has 8 single-input
+        # "bit{i}.match" gates that fire when bit i of v matches bit i of k,
+        # ANDed by ".all". Final ".weight" ORs all 52 "all" outputs.
+        bits = int_to_bits_msb(v, 8)
+        ks = [k for k in range(256) if k % 5 == 0]
+        alls = []
+        for k in ks:
+            matches = [gate(f"modular.mod5.eq.k{k}.bit{i}.match", [bits[i]]) for i in range(8)]
+            alls.append(gate(f"modular.mod5.eq.k{k}.all", matches))
+        return gate("modular.mod5", alls)
+
+    hits = [v for v in range(256) if mod5(v)]
+    print(f"  v in [0,255] with mod5(v)==1: {len(hits)} hits, first 12: {hits[:12]}")
+    print(f"  Sanity (each %5): {[h % 5 for h in hits[:12]]}")
+    print()
+
+    # ---------- Demo 4: CPU running an assembled program ----------
+    if args.skip_cpu or MEM_BYTES < 0x84:
+        if args.skip_cpu:
+            print("Demo 4 skipped (--skip-cpu).")
+        else:
+            print(f"Demo 4 skipped (memory={MEM_BYTES}B too small for the demo program).")
+        return 0
+
+    print("=" * 64)
+    print(f" Demo 4: Threshold CPU running an assembled program ({MEM_BYTES} B memory)")
+    print("=" * 64)
+    print("  Program: sum 5+4+3+2+1 via loop")
+    print("           uses LOAD/STORE/ADD/SUB/CMP/JNZ/HALT, all threshold-gated")
+    print("  Running ... (larger memories take longer because every memory access")
+    print("              decodes against every address line)")
+    cpu = GenericThresholdCPU({k: v for k, v in T.items()})
+    mem, expected = builtin_program(ADDR_BITS)
+    state = {"pc": 0, "regs": [0] * 4, "flags": [0] * 4, "mem": mem, "halted": False}
+    final, cycles = cpu.run(state, max_cycles=200)
+    got = final["mem"][0x83]
+    print(f"  Halted after {cycles} cycles")
+    print(f"  R0={final['regs'][0]} R1={final['regs'][1]} "
+          f"R2={final['regs'][2]} R3={final['regs'][3]}")
+    print(f"  M[0x0083] = {got}  (expected {expected})  [{'OK' if got == expected else 'FAIL'}]")
+    return 0 if got == expected else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

prune_weights.py

@@ -0,0 +1,484 @@
+"""
+BATCHED WEIGHT PRUNING (GPU-optimized)
+======================================
+Phase 1: Batch eval all candidates in parallel
+Phase 2: Apply all successes at once, binary search if conflicts
+"""
+
+import argparse
+import os
+import time
+
+import torch
+from safetensors.torch import save_file
+from eval import BatchedFitnessEvaluator, create_population, load_model
+
+torch.manual_seed(0)
+
+DEFAULT_OUTPUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "pruned.safetensors")
+
+
+def format_time(seconds):
+    if seconds < 60:
+        return f"{seconds:.1f}s"
+    elif seconds < 3600:
+        return f"{seconds/60:.1f}m"
+    else:
+        return f"{seconds/3600:.1f}h"
+
+
+def format_eta(elapsed, done, total):
+    if done == 0:
+        return "calculating..."
+    rate = done / elapsed
+    remaining = (total - done) / rate
+    return format_time(remaining)
+
+
+def apply_reductions(model, reductions):
+    """Apply a list of (name, flat_idx, shape, old_val) reductions."""
+    for name, flat_idx, shape, old_val in reductions:
+        new_val = old_val - 1 if old_val > 0 else old_val + 1
+        flat = model[name].flatten()
+        if flat[flat_idx].item() == old_val:
+            flat[flat_idx] = new_val
+            model[name] = flat.view(shape)
+
+
+def revert_reductions(model, reductions):
+    """Revert a list of reductions."""
+    for name, flat_idx, shape, old_val in reductions:
+        flat = model[name].flatten()
+        new_val = old_val - 1 if old_val > 0 else old_val + 1
+        if flat[flat_idx].item() == new_val:
+            flat[flat_idx] = old_val
+            model[name] = flat.view(shape)
+
+
+def check_fitness(model, evaluator, device):
+    """Check model fitness."""
+    torch.manual_seed(0)
+    pop = create_population(model, 1, device)
+    return evaluator.evaluate(pop, debug=False)[0].item()
+
+
+def sequential_conflict_resolution(model, evaluator, device, candidates, base_magnitude):
+    """
+    Sequential fallback - tests and applies reductions one at a time.
+    Slower but guarantees no interaction bugs.
+    """
+    accepted = []
+    for i, (name, flat_idx, shape, old_val) in enumerate(candidates):
+        apply_reductions(model, [(name, flat_idx, shape, old_val)])
+        fitness = check_fitness(model, evaluator, device)
+        if fitness >= 0.9999:
+            accepted.append((name, flat_idx, shape, old_val))
+            if (i + 1) % 50 == 0:
+                current_mag = sum(t.abs().sum().item() for t in model.values())
+                reduction_pct = 100 * (1 - current_mag / base_magnitude)
+                print(f"          Sequential: {len(accepted)}/{i+1} accepted | mag={current_mag:.0f} (-{reduction_pct:.2f}%)")
+        else:
+            revert_reductions(model, [(name, flat_idx, shape, old_val)])
+    return accepted
+
+
+def batched_conflict_resolution(model, evaluator, device, candidates, base_magnitude):
+    """
+    Batched binary search - evaluates multiple branches in parallel.
+    Uses BFS instead of DFS to maximize batching opportunities.
+    Verifies cumulative effect after each batch to prevent interaction bugs.
+    """
+    if len(candidates) == 0:
+        return []
+
+    # First try all at once
+    print(f"        Trying {len(candidates)} reductions at once...")
+    apply_reductions(model, candidates)
+    fitness = check_fitness(model, evaluator, device)
+
+    if fitness >= 0.9999:
+        current_mag = sum(t.abs().sum().item() for t in model.values())
+        reduction_pct = 100 * (1 - current_mag / base_magnitude)
+        print(f"        ALL {len(candidates)} OK | fitness={fitness:.6f} | "
+              f"mag={current_mag:.0f} (-{reduction_pct:.2f}%)")
+        return candidates
+
+    # Conflict - revert and use batched BFS
+    revert_reductions(model, candidates)
+    print(f"        CONFLICT (fitness={fitness:.6f}), starting batched resolution...")
+
+    accepted = []
+    # Queue of (candidate_list, depth) to process
+    pending = [(candidates, 0)]
+
+    while pending:
+        # Collect all pending groups for batch evaluation
+        to_eval = []
+        for group, depth in pending:
+            if len(group) == 0:
+                continue
+            elif len(group) == 1:
+                to_eval.append((group, depth, 'single'))
+            else:
+                to_eval.append((group, depth, 'group'))
+
+        pending = []
+
+        if not to_eval:
+            break
+
+        # Build batch: create model variants for each group
+        batch_size = len(to_eval)
+        print(f"          Batch evaluating {batch_size} groups...")
+
+        # Create population for batch eval
+        pop = {}
+        for name, tensor in model.items():
+            pop[name] = tensor.unsqueeze(0).expand(batch_size, *tensor.shape).clone().to(device)
+
+        # Apply each group's reductions to its population slot
+        for idx, (group, depth, gtype) in enumerate(to_eval):
+            for name, flat_idx, shape, old_val in group:
+                new_val = old_val - 1 if old_val > 0 else old_val + 1
+                flat_view = pop[name][idx].flatten()
+                # Check if not already modified in base model
+                base_val = model[name].flatten()[flat_idx].item()
+                if base_val == old_val:
+                    flat_view[flat_idx] = new_val
+
+        # Batch evaluate
+        torch.manual_seed(0)
+        fitnesses = evaluator.evaluate(pop, debug=False)
+
+        # Process results - collect accepted groups first, then verify
+        batch_accepted = []
+        ok_count = 0
+        conflict_count = 0
+        fail_count = 0
+
+        for idx, (group, depth, gtype) in enumerate(to_eval):
+            fit = fitnesses[idx].item()
+            indent = "          " + "  " * depth
+
+            if fit >= 0.9999:
+                batch_accepted.append((group, depth, indent))
+                ok_count += len(group)
+            else:
+                if len(group) == 1:
+                    name, flat_idx, shape, old_val = group[0]
+                    print(f"{indent}[1/1] FAIL {name}[{flat_idx}] | fitness={fit:.6f}")
+                    fail_count += 1
+                else:
+                    mid = len(group) // 2
+                    left = group[:mid]
+                    right = group[mid:]
+                    print(f"{indent}CONFLICT ({len(group)}) fitness={fit:.6f} -> split {len(left)}+{len(right)}")
+                    pending.append((left, depth + 1))
+                    pending.append((right, depth + 1))
+                    conflict_count += 1
+
+        # Apply all batch-accepted reductions
+        all_batch_reductions = []
+        for group, depth, indent in batch_accepted:
+            apply_reductions(model, group)
+            all_batch_reductions.extend(group)
+
+        # Verify cumulative effect
+        if all_batch_reductions:
+            verify_fitness = check_fitness(model, evaluator, device)
+            if verify_fitness >= 0.9999:
+                # All good - commit these reductions
+                for group, depth, indent in batch_accepted:
+                    current_mag = sum(t.abs().sum().item() for t in model.values())
+                    reduction_pct = 100 * (1 - current_mag / base_magnitude)
+                    if len(group) == 1:
+                        name, flat_idx, shape, old_val = group[0]
+                        print(f"{indent}[1/1] OK {name}[{flat_idx}] | mag={current_mag:.0f} (-{reduction_pct:.2f}%)")
+                    else:
+                        print(f"{indent}ALL {len(group)} OK | mag={current_mag:.0f} (-{reduction_pct:.2f}%)")
+                accepted.extend(all_batch_reductions)
+                print(f"          Batch result: {ok_count} accepted, {conflict_count} split, {fail_count} failed")
+            else:
+                # Interaction bug detected - revert and use sequential fallback
+                print(f"          INTERACTION BUG detected (batch fitness={verify_fitness:.6f})")
+                print(f"          Reverting {len(all_batch_reductions)} reductions, falling back to sequential...")
+                revert_reductions(model, all_batch_reductions)
+
+                # Process each group sequentially
+                seq_accepted = sequential_conflict_resolution(
+                    model, evaluator, device, all_batch_reductions, base_magnitude
+                )
+                accepted.extend(seq_accepted)
+                print(f"          Sequential fallback: {len(seq_accepted)}/{len(all_batch_reductions)} accepted")
+        else:
+            print(f"          Batch result: {ok_count} accepted, {conflict_count} split, {fail_count} failed")
+
+    return accepted
+
+
+def prune_weights(
+    passes: int = 10,
+    batch_size: int = 5000,
+    device: str = 'cuda',
+    checkpoint_path: str = DEFAULT_OUTPUT
+):
+    print("=" * 80)
+    print(" BATCHED WEIGHT PRUNING (GPU-optimized)")
+    print("=" * 80)
+    print(f" Device: {device}")
+    print(f" Batch size: {batch_size}")
+    print(f" Max passes: {passes}")
+    print("=" * 80)
+
+    # Load model
+    print("\n[1/4] LOADING MODEL...")
+    load_start = time.perf_counter()
+    model = load_model()
+    load_time = time.perf_counter() - load_start
+
+    n_params = sum(t.numel() for t in model.values())
+    n_tensors = len(model)
+    base_magnitude = sum(t.abs().sum().item() for t in model.values())
+    base_max = max(t.abs().max().item() for t in model.values())
+    nonzero_params = sum((t != 0).sum().item() for t in model.values())
+
+    print(f"       Loaded in {load_time:.2f}s")
+    print(f"       Tensors: {n_tensors}")
+    print(f"       Parameters: {n_params}")
+    print(f"       Non-zero parameters: {nonzero_params}")
+    print(f"       Total magnitude: {base_magnitude:.0f}")
+    print(f"       Max weight: {base_max:.0f}")
+
+    # Initialize evaluator
+    print("\n[2/4] INITIALIZING EVALUATOR...")
+    eval_start = time.perf_counter()
+    evaluator = BatchedFitnessEvaluator(device=device)
+    eval_time = time.perf_counter() - eval_start
+    print(f"       Initialized in {eval_time:.2f}s")
+
+    # Verify initial fitness
+    print("\n[3/4] VERIFYING BASE MODEL...")
+    initial_fitness = check_fitness(model, evaluator, device)
+    print(f"       Fitness: {initial_fitness:.6f}")
+
+    if initial_fitness < 0.9999:
+        print(f"       ERROR: Base model fitness {initial_fitness:.6f} < 0.9999")
+        return None
+
+    print(f"       STATUS: PASS")
+
+    # Build parameter list
+    print("\n[4/4] BUILDING PARAMETER INDEX...")
+    param_list = []
+    for name, tensor in model.items():
+        flat = tensor.flatten()
+        for i in range(len(flat)):
+            param_list.append((name, i, tensor.shape))
+    print(f"       Indexed {len(param_list)} parameters")
+
+    # Main pruning loop
+    print("\n" + "=" * 80)
+    print(" PRUNING STARTED")
+    print("=" * 80)
+
+    total_reductions = 0
+    pruning_start = time.perf_counter()
+
+    for pass_num in range(passes):
+        torch.manual_seed(0)
+        pass_start = time.perf_counter()
+
+        print(f"\n{'='*80}")
+        print(f" PASS {pass_num + 1}/{passes}")
+        print(f"{'='*80}")
+
+        # Count candidates
+        candidates = []
+        for name, idx, shape in param_list:
+            flat = model[name].flatten()
+            val = flat[idx].item()
+            if val != 0:
+                candidates.append((name, idx, shape, val))
+
+        n_candidates = len(candidates)
+        print(f"\n    Candidates: {n_candidates} non-zero weights")
+
+        if n_candidates == 0:
+            print(f"    No candidates remaining. Stopping.")
+            break
+
+        # Phase 1: Batch evaluation
+        print(f"\n    PHASE 1: Batch evaluation (testing each reduction independently)")
+        print(f"    " + "-" * 60)
+        phase1_start = time.perf_counter()
+        successful_candidates = []
+        n_batches = (n_candidates + batch_size - 1) // batch_size
+
+        for batch_idx, batch_start_idx in enumerate(range(0, n_candidates, batch_size)):
+            batch = candidates[batch_start_idx:batch_start_idx + batch_size]
+            batch_len = len(batch)
+            batch_start_time = time.perf_counter()
+
+            # Build population
+            pop = {}
+            for name, tensor in model.items():
+                pop[name] = tensor.unsqueeze(0).expand(batch_len, *tensor.shape).clone().to(device)
+
+            # Apply reductions
+            for pop_idx, (name, flat_idx, shape, old_val) in enumerate(batch):
+                new_val = old_val - 1 if old_val > 0 else old_val + 1
+                flat_view = pop[name][pop_idx].flatten()
+                flat_view[flat_idx] = new_val
+
+            # Evaluate
+            torch.manual_seed(0)
+            if device == 'cuda':
+                torch.cuda.synchronize()
+            fitness = evaluator.evaluate(pop, debug=False)
+            if device == 'cuda':
+                torch.cuda.synchronize()
+
+            # Collect successes
+            batch_successes = 0
+            for pop_idx, (name, flat_idx, shape, old_val) in enumerate(batch):
+                if fitness[pop_idx].item() >= 0.9999:
+                    successful_candidates.append((name, flat_idx, shape, old_val))
+                    batch_successes += 1
+
+            batch_time = time.perf_counter() - batch_start_time
+            elapsed = time.perf_counter() - phase1_start
+            done = batch_start_idx + batch_len
+            eta = format_eta(elapsed, done, n_candidates)
+            throughput = batch_len / batch_time
+
+            print(f"      Batch {batch_idx + 1}/{n_batches}: "
+                  f"{batch_successes}/{batch_len} passed ({100*batch_successes/batch_len:.1f}%) | "
+                  f"Total OK: {len(successful_candidates)} | "
+                  f"Progress: {done}/{n_candidates} ({100*done/n_candidates:.1f}%) | "
+                  f"Speed: {throughput:.0f}/s | "
+                  f"ETA: {eta}")
+
+        phase1_time = time.perf_counter() - phase1_start
+        print(f"\n    Phase 1 complete: {len(successful_candidates)}/{n_candidates} candidates "
+              f"({100*len(successful_candidates)/n_candidates:.1f}%) in {format_time(phase1_time)}")
+
+        # Phase 2: Apply with conflict resolution
+        if len(successful_candidates) == 0:
+            print(f"\n    No reductions possible. Stopping.")
+            break
+
+        print(f"\n    PHASE 2: Apply reductions with conflict resolution")
+        print(f"    " + "-" * 60)
+        phase2_start = time.perf_counter()
+
+        accepted = batched_conflict_resolution(model, evaluator, device, successful_candidates, base_magnitude)
+        pass_reductions = len(accepted)
+
+        phase2_time = time.perf_counter() - phase2_start
+        print(f"\n    Phase 2 complete: {pass_reductions} reductions applied in {format_time(phase2_time)}")
+
+        # Pass summary
+        total_reductions += pass_reductions
+        current_magnitude = sum(t.abs().sum().item() for t in model.values())
+        current_nonzero = sum((t != 0).sum().item() for t in model.values())
+        pass_time = time.perf_counter() - pass_start
+        reduction_pct = 100 * (1 - current_magnitude / base_magnitude)
+
+        print(f"\n    PASS {pass_num + 1} SUMMARY:")
+        print(f"      Reductions this pass: {pass_reductions}")
+        print(f"      Total reductions: {total_reductions}")
+        print(f"      Current magnitude: {current_magnitude:.0f} (-{reduction_pct:.2f}%)")
+        print(f"      Current non-zero: {current_nonzero}")
+        print(f"      Pass time: {format_time(pass_time)}")
+
+        # Verify after pass
+        print(f"\n    Verifying model integrity...")
+        fitness = check_fitness(model, evaluator, device)
+        print(f"      Fitness: {fitness:.6f} {'PASS' if fitness >= 0.9999 else 'FAIL'}")
+
+        # Save checkpoint after each pass
+        checkpoint_name = checkpoint_path.replace('.safetensors', f'_pass{pass_num + 1}.safetensors')
+        print(f"\n    Saving checkpoint: {checkpoint_name}")
+        save_file(model, checkpoint_name)
+        print(f"      Saved. Magnitude: {current_magnitude:.0f} (-{reduction_pct:.2f}%)")
+
+        # Also save as "latest" for easy access
+        latest_path = checkpoint_path.replace('.safetensors', '_latest.safetensors')
+        save_file(model, latest_path)
+        print(f"      Also saved as: {latest_path}")
+
+        if pass_reductions == 0:
+            print(f"\n    No reductions achieved. Stopping early.")
+            break
+
+    # Final summary
+    pruning_time = time.perf_counter() - pruning_start
+    final_magnitude = sum(t.abs().sum().item() for t in model.values())
+    final_max = max(t.abs().max().item() for t in model.values())
+    final_nonzero = sum((t != 0).sum().item() for t in model.values())
+    reduction_pct = 100 * (1 - final_magnitude / base_magnitude)
+
+    print("\n" + "=" * 80)
+    print(" PRUNING COMPLETE")
+    print("=" * 80)
+    print(f"\n RESULTS:")
+    print(f"   Original magnitude:  {base_magnitude:.0f}")
+    print(f"   Final magnitude:     {final_magnitude:.0f}")
+    print(f"   Reduction:           {reduction_pct:.2f}%")
+    print(f"   Total reductions:    {total_reductions}")
+    print(f"   Original non-zero:   {nonzero_params}")
+    print(f"   Final non-zero:      {final_nonzero}")
+    print(f"   Zeros created:       {nonzero_params - final_nonzero}")
+    print(f"   Max weight:          {final_max:.0f}")
+    print(f"   Total time:          {format_time(pruning_time)}")
+
+    # Save
+    print(f"\n SAVING to {checkpoint_path}...")
+    save_file(model, checkpoint_path)
+    print(f"   Saved.")
+
+    # Final verification
+    print(f"\n FINAL VERIFICATION...")
+    from safetensors import safe_open
+    f = safe_open(checkpoint_path, framework='numpy')
+    verify_model = {name: torch.tensor(f.get_tensor(name)).float() for name in f.keys()}
+    verify_fitness = check_fitness(verify_model, evaluator, device)
+    print(f"   Fitness: {verify_fitness:.6f}")
+
+    if verify_fitness >= 0.9999:
+        print(f"   STATUS: PASS")
+    else:
+        print(f"   STATUS: FAIL - Model corrupted!")
+
+    print("\n" + "=" * 80)
+    return model
+
+
+MAX_BATCH_SIZE = 80000
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description='Batched Weight Pruning')
+    parser.add_argument('--passes', type=int, default=10,
+                        help='Maximum pruning passes (default: 10)')
+    parser.add_argument('--batch_size', type=int, default=80000,
+                        help=f'Batch size for parallel evaluation (default: 80000, max: {MAX_BATCH_SIZE})')
+    parser.add_argument('--device', type=str, default='cuda',
+                        help='Device: cuda or cpu (default: cuda)')
+    parser.add_argument('--output', type=str, default=DEFAULT_OUTPUT,
+                        help='Output path')
+    args = parser.parse_args()
+
+    if args.batch_size > MAX_BATCH_SIZE:
+        print(f"WARNING: batch_size {args.batch_size} exceeds maximum {MAX_BATCH_SIZE}. Clamping.")
+        args.batch_size = MAX_BATCH_SIZE
+
+    print(f"\nStarting at {time.strftime('%Y-%m-%d %H:%M:%S')}\n")
+
+    prune_weights(
+        passes=args.passes,
+        batch_size=args.batch_size,
+        device=args.device,
+        checkpoint_path=args.output
+    )
+
+    print(f"\nFinished at {time.strftime('%Y-%m-%d %H:%M:%S')}")

quantize.py

@@ -0,0 +1,406 @@
+"""
+Quantize threshold-computer safetensors to the minimum signed integer
+dtype that exactly represents each tensor.
+
+Weights and biases in this library are integer-valued by construction,
+with one historical exception: a handful of legacy buffer gates use a
+bias of -0.5 (e.g. arithmetic.asr8bit.bit*.bias). For binary inputs,
+H(x - 0.5) and H(x - 1) are identical, so those biases are floored to
+-1 before casting.
+
+This is a packaging optimization, not a precision change: the eval
+pipeline already promotes weights to float32 on load, so integer
+storage is exact.
+
+The --ternary flag also rewrites single-input weight=+/-2 identity
+buffers (SHL/SHR/ROL/ROR bit gates, stack data buffers, RET address
+buffers, flag buffers) to weight=+/-1 with bias adjusted as needed to
+preserve heaviside output for binary inputs. After this pass every
+weight tensor in the file lies in {-1, 0, 1} except for positional
+comparators and a few hand-constructed modular arithmetic circuits
+(see the violation report); fully ternarizing those requires bit-
+cascading in build.py.
+
+Usage:
+    python quantize.py path/to/file.safetensors                      # in-place
+    python quantize.py path/to/file.safetensors -o out.safetensors   # to new file
+    python quantize.py variants/                                      # whole directory in place
+    python quantize.py variants/ -o variants_int/                     # whole directory to new dir
+    python quantize.py file.safetensors --ternary                     # try ternary weights
+    python quantize.py file.safetensors --ternary --strict            # error if any weight non-ternary
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+from typing import Dict, Tuple
+
+import torch
+from safetensors import safe_open
+from safetensors.torch import save_file
+
+DTYPES = [
+    (torch.int8, -(1 << 7), (1 << 7) - 1),
+    (torch.int16, -(1 << 15), (1 << 15) - 1),
+    (torch.int32, -(1 << 31), (1 << 31) - 1),
+    (torch.int64, None, None),  # always fits
+]
+
+
+def _normalize_to_int(tensor: torch.Tensor) -> torch.Tensor:
+    """Return a tensor with strictly integer values, floored from any
+    half-integer values. Floor (not round) because a -0.5 bias must
+    become -1 (not 0) to preserve H(x + bias) for binary x."""
+    if not tensor.dtype.is_floating_point:
+        return tensor.to(torch.float64)  # promote for range checks
+    tf = tensor.to(torch.float64)
+    rounded = tf.round()
+    if torch.equal(rounded, tf):
+        return tf
+    doubled = tf * 2.0
+    if torch.equal(doubled.round(), doubled):
+        return torch.floor(tf)
+    raise ValueError(
+        f"tensor has non-half-integer values; range "
+        f"[{tf.min().item()}, {tf.max().item()}]"
+    )
+
+
+def _min_signed_int_dtype(tensor: torch.Tensor) -> torch.dtype:
+    if tensor.numel() == 0:
+        return torch.int8
+    lo = int(tensor.min().item())
+    hi = int(tensor.max().item())
+    for dtype, lo_lim, hi_lim in DTYPES:
+        if lo_lim is None or (lo_lim <= lo and hi <= hi_lim):
+            return dtype
+    return torch.int64
+
+
+def _ternarize_modular_and_patterns(
+    tensors: Dict[str, torch.Tensor],
+) -> Tuple[Dict[str, torch.Tensor], Dict]:
+    """Replace seed-file modular detectors and pattern_recognition gates
+    with bit-cascade-equivalent ternary structures.
+
+    Modular: for each modulus N in {3,5,6,7,9,10,11,12}, replace the
+    layer1.geq{i}/layer1.leq{i}/layer2.eq{i}/layer3.or chain with a
+    bit-cascade equality detector per multiple of N in [0, 256), then
+    OR all detectors together. Top-level gate stays named
+    `modular.mod{N}` (a multi-input OR over per-multiple matches).
+
+    Pattern_recognition.leadingones / trailingones are dropped: they are
+    seed-file artifacts with no eval coverage and no downstream
+    consumers in this codebase.
+
+    Moduli whose bit-cascade structure is already present are left
+    untouched (including their .inputs routing metadata); the rebuild
+    only fires on the legacy layer1/layer2/layer3 layout.
+    """
+    new_tensors = dict(tensors)
+
+    # --- pattern_recognition: drop leadingones/trailingones ---
+    pr_dropped = 0
+    for k in list(new_tensors.keys()):
+        if (k.startswith("pattern_recognition.leadingones")
+                or k.startswith("pattern_recognition.trailingones")):
+            del new_tensors[k]
+            pr_dropped += 1
+
+    # --- modular: rebuild as ternary bit-cascade equality per multiple ---
+    moduli = [3, 5, 6, 7, 9, 10, 11, 12]
+    mod_gates_added = 0
+    for mod in moduli:
+        prefix = f"modular.mod{mod}"
+        if f"{prefix}.eq.k0.all.weight" in new_tensors:
+            continue  # already bit-cascade; rebuilding would only shed metadata
+        # Drop old structure
+        for k in list(new_tensors.keys()):
+            if k.startswith(prefix + "."):
+                del new_tensors[k]
+
+        multiples = list(range(0, 256, mod))
+        # Per-bit match gates + per-multiple AND
+        for k in multiples:
+            for i in range(8):  # i=0 is MSB (matches inputs MSB-first ordering)
+                k_bit = (k >> (7 - i)) & 1
+                if k_bit == 1:
+                    # bit_match = x[i]: H(x - 0.5) ~~ identity for binary;
+                    # use weight=1, bias=-1 -> H(x-1): x=0 -> 0, x=1 -> 1.
+                    new_tensors[f"{prefix}.eq.k{k}.bit{i}.match.weight"] = torch.tensor([1.0], dtype=torch.float64)
+                    new_tensors[f"{prefix}.eq.k{k}.bit{i}.match.bias"] = torch.tensor([-1.0], dtype=torch.float64)
+                else:
+                    # bit_match = NOT x[i]: weight=-1, bias=0 -> H(-x): x=0 -> 1, x=1 -> 0.
+                    new_tensors[f"{prefix}.eq.k{k}.bit{i}.match.weight"] = torch.tensor([-1.0], dtype=torch.float64)
+                    new_tensors[f"{prefix}.eq.k{k}.bit{i}.match.bias"] = torch.tensor([0.0], dtype=torch.float64)
+                mod_gates_added += 1
+            # AND of all 8 bit-matches: weights [1]*8, bias -8
+            new_tensors[f"{prefix}.eq.k{k}.all.weight"] = torch.tensor([1.0] * 8, dtype=torch.float64)
+            new_tensors[f"{prefix}.eq.k{k}.all.bias"] = torch.tensor([-8.0], dtype=torch.float64)
+            mod_gates_added += 1
+
+        # Final OR over all per-multiple match outputs
+        m = len(multiples)
+        new_tensors[f"{prefix}.weight"] = torch.tensor([1.0] * m, dtype=torch.float64)
+        new_tensors[f"{prefix}.bias"] = torch.tensor([-1.0], dtype=torch.float64)
+        mod_gates_added += 1
+
+    return new_tensors, {
+        "pattern_recognition_dropped": pr_dropped,
+        "modular_gates_added": mod_gates_added,
+        "modular_moduli": len(moduli),
+    }
+
+
+def _ternarize_buffers(
+    tensors: Dict[str, torch.Tensor],
+) -> Tuple[Dict[str, torch.Tensor], Dict]:
+    """Rewrite single-input weight=+-2 identity buffers as weight=+-1 with
+    bias adjusted to preserve heaviside output for binary inputs.
+
+    For a single-input gate H(w*x + b) with x in {0, 1}, the only thing
+    that matters is the pair (H(b), H(w + b)). Pick the smallest integer
+    bias b' such that (H(b'), H(sgn + b')) matches, with sgn = sign(w).
+
+    Returns (new_tensors, stats). stats has 'fixed', 'failed', 'failed_names'.
+    """
+    new_tensors = dict(tensors)
+    fixed = 0
+    failed_names = []
+
+    weight_keys = [k for k in tensors if k.endswith(".weight")]
+    for wkey in weight_keys:
+        w = tensors[wkey]
+        wf = w.float() if w.dtype.is_floating_point else w.to(torch.float64).float()
+        if (wf.abs() <= 1.0).all():
+            continue  # already ternary
+
+        gate = wkey[: -len(".weight")]
+        bkey = gate + ".bias"
+
+        # Single-input weight=+-2 buffer with single bias
+        if (
+            wf.numel() == 1
+            and abs(wf.item()) == 2.0
+            and bkey in tensors
+            and tensors[bkey].numel() == 1
+        ):
+            w_val = wf.item()
+            b_val = float(tensors[bkey].float().item())
+            sgn = 1.0 if w_val > 0 else -1.0
+            x0_target = 1 if b_val >= 0 else 0
+            x1_target = 1 if (w_val + b_val) >= 0 else 0
+            chosen = None
+            # Prefer keeping the bias unchanged when possible
+            for b_new in [int(b_val), int(b_val) - 1, -1, 0, -2, 1, -3, 2]:
+                x0 = 1 if b_new >= 0 else 0
+                x1 = 1 if (sgn + b_new) >= 0 else 0
+                if x0 == x0_target and x1 == x1_target:
+                    chosen = b_new
+                    break
+            if chosen is not None:
+                new_tensors[wkey] = torch.tensor([sgn], dtype=torch.float64)
+                new_tensors[bkey] = torch.tensor([float(chosen)], dtype=torch.float64)
+                fixed += 1
+                continue
+
+        failed_names.append(wkey)
+
+    return new_tensors, {"fixed": fixed, "failed_names": failed_names}
+
+
+def quantize_tensors(
+    tensors: Dict[str, torch.Tensor],
+    ternary: bool = False,
+) -> Tuple[Dict[str, torch.Tensor], Dict[str, int], Tuple[int, int], Dict]:
+    """Quantize a dict of tensors. Returns
+    (new_tensors, dtype_counts, (bytes_before, bytes_after), ternary_stats)."""
+    ternary_stats: Dict = {"applied": False, "fixed": 0, "failed_names": []}
+    if ternary:
+        tensors, ternary_stats = _ternarize_buffers(tensors)
+        ternary_stats["applied"] = True
+        # Also rebuild modular detectors and drop pattern_recognition stragglers.
+        tensors, mod_stats = _ternarize_modular_and_patterns(tensors)
+        ternary_stats["modular_gates_added"] = mod_stats["modular_gates_added"]
+        ternary_stats["pattern_recognition_dropped"] = mod_stats["pattern_recognition_dropped"]
+
+    new_tensors: Dict[str, torch.Tensor] = {}
+    counts: Dict[str, int] = {"int8": 0, "int16": 0, "int32": 0, "int64": 0,
+                              "manifest_kept": 0, "skipped": 0}
+    bytes_before = 0
+    bytes_after = 0
+
+    for name, t in tensors.items():
+        bytes_before += t.numel() * t.element_size()
+
+        if name.startswith("manifest."):
+            new_tensors[name] = t
+            counts["manifest_kept"] += 1
+            bytes_after += t.numel() * t.element_size()
+            continue
+
+        try:
+            normalized = _normalize_to_int(t)
+        except ValueError:
+            new_tensors[name] = t
+            counts["skipped"] += 1
+            bytes_after += t.numel() * t.element_size()
+            continue
+
+        target = _min_signed_int_dtype(normalized)
+        cast = normalized.to(target)
+        new_tensors[name] = cast
+        bytes_after += cast.numel() * cast.element_size()
+        counts[str(target).replace("torch.", "")] += 1
+
+    return new_tensors, counts, (bytes_before, bytes_after), ternary_stats
+
+
+def quantize_file(in_path: Path, out_path: Path, verbose: bool = False,
+                  ternary: bool = False, strict_ternary: bool = False) -> Dict:
+    file_before = in_path.stat().st_size
+    tensors: Dict[str, torch.Tensor] = {}
+    metadata: Dict[str, str] = {}
+    with safe_open(str(in_path), framework="pt") as f:
+        meta = f.metadata()
+        if meta:
+            metadata = dict(meta)
+        for name in f.keys():
+            # clone so the source mmap can be released before we write
+            tensors[name] = f.get_tensor(name).clone()
+
+    new_tensors, counts, (before, after), tstats = quantize_tensors(tensors, ternary=ternary)
+
+    # Audit final ternary status (count of weight tensors with |w| > 1)
+    final_nonternary = []
+    for k, v in new_tensors.items():
+        if not k.endswith(".weight"):
+            continue
+        if k.startswith("manifest."):
+            continue
+        vf = v.float() if v.dtype.is_floating_point else v.to(torch.float64).float()
+        if (vf.abs() > 1.0).any():
+            final_nonternary.append(k)
+
+    if ternary and strict_ternary and final_nonternary:
+        raise ValueError(
+            f"--strict failed: {len(final_nonternary)} weight tensors are not "
+            f"ternary after transformation; first: {final_nonternary[:5]}"
+        )
+
+    # Drop the original mmap-backed tensors before writing in-place.
+    del tensors
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    if ternary:
+        # Note ternary mode in metadata so downstream tools can see it
+        if metadata is None:
+            metadata = {}
+        metadata = dict(metadata)
+        metadata["weight_quantization"] = (
+            "ternary_partial" if final_nonternary else "ternary"
+        )
+    save_file(new_tensors, str(out_path), metadata=metadata or None)
+
+    file_after = out_path.stat().st_size
+    return {
+        "in_path": str(in_path),
+        "out_path": str(out_path),
+        "tensor_counts": counts,
+        "tensor_bytes_before": before,
+        "tensor_bytes_after": after,
+        "file_size_before": file_before,
+        "file_size_after": file_after,
+        "ternary": tstats,
+        "final_nonternary": final_nonternary,
+    }
+
+
+def _print_summary(label: str, info: Dict) -> None:
+    cb = info["tensor_bytes_before"]
+    ca = info["tensor_bytes_after"]
+    fb = info["file_size_before"]
+    fa = info["file_size_after"]
+    counts = info["tensor_counts"]
+    bucket_str = "  ".join(f"{k}={v}" for k, v in counts.items() if v)
+    ratio_t = cb / ca if ca else 1.0
+    ratio_f = fb / fa if fa else 1.0
+    print(
+        f"  {label}: file {fb / 1e6:6.1f} MB -> {fa / 1e6:6.1f} MB "
+        f"({ratio_f:.2f}x);  tensor data {cb / 1e6:6.1f} MB -> {ca / 1e6:6.1f} MB "
+        f"({ratio_t:.2f}x)"
+    )
+    print(f"      {bucket_str}")
+    if info.get("ternary", {}).get("applied"):
+        ts = info["ternary"]
+        nt = info["final_nonternary"]
+        print(f"      ternary: {ts['fixed']} buffer gates rewritten; "
+              f"{len(nt)} weight tensors remain non-ternary")
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Quantize safetensors to min signed int dtype")
+    parser.add_argument("input", type=Path, help=".safetensors file or directory of files")
+    parser.add_argument("-o", "--output", type=Path, default=None,
+                        help="output file or directory (default: in-place)")
+    parser.add_argument("-v", "--verbose", action="store_true")
+    parser.add_argument("--ternary", action="store_true",
+                        help="Rewrite single-input weight=+/-2 buffers as +/-1 to push toward "
+                             "ternary {-1, 0, 1} weights and report remaining violations")
+    parser.add_argument("--strict", action="store_true",
+                        help="With --ternary, fail if any weight tensor is still non-ternary")
+    parser.add_argument("--report-violations", type=int, default=0, metavar="N",
+                        help="Print first N non-ternary weight tensor names per file")
+    args = parser.parse_args()
+
+    inputs = []
+    if args.input.is_dir():
+        inputs = sorted(p for p in args.input.glob("*.safetensors"))
+    elif args.input.is_file():
+        inputs = [args.input]
+    else:
+        print(f"not found: {args.input}", file=sys.stderr)
+        return 2
+
+    if not inputs:
+        print(f"no .safetensors files under {args.input}", file=sys.stderr)
+        return 2
+
+    if args.output is None:
+        outputs = inputs  # in-place
+    elif args.output.suffix == ".safetensors":
+        if len(inputs) != 1:
+            print("output is a single file but input is a directory; pass a directory output", file=sys.stderr)
+            return 2
+        outputs = [args.output]
+    else:
+        args.output.mkdir(parents=True, exist_ok=True)
+        outputs = [args.output / p.name for p in inputs]
+
+    total_before = 0
+    total_after = 0
+    print(f"Quantizing {len(inputs)} file(s)" + (" (ternary mode)" if args.ternary else "") + "\n")
+    for src, dst in zip(inputs, outputs):
+        info = quantize_file(src, dst, verbose=args.verbose,
+                             ternary=args.ternary, strict_ternary=args.strict)
+        _print_summary(src.name, info)
+        if args.report_violations and info.get("final_nonternary"):
+            for name in info["final_nonternary"][: args.report_violations]:
+                print(f"        non-ternary: {name}")
+            if len(info["final_nonternary"]) > args.report_violations:
+                print(f"        ... and {len(info['final_nonternary']) - args.report_violations} more")
+        total_before += info["file_size_before"]
+        total_after += info["file_size_after"]
+
+    print()
+    print("=" * 76)
+    print(
+        f"Total: {total_before / 1e6:.1f} MB -> {total_after / 1e6:.1f} MB "
+        f"({total_before / max(total_after, 1):.2f}x reduction)"
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

test_cpu.py

@@ -0,0 +1,138 @@
+"""
+Run the CPU program suite against a threshold-computer variant.
+
+Loads weights, instantiates GenericThresholdCPU, runs each program from
+cpu_programs.SUITE, and verifies expected memory contents at HALT.
+
+Usage:
+    python test_cpu.py                                       # default: 1KB variant, fast
+    python test_cpu.py --model neural_computer.safetensors   # 64KB canonical, slow
+    python test_cpu.py --only fib,sum_n                      # subset of suite
+"""
+
+from __future__ import annotations
+import argparse
+import os
+import sys
+import time
+from pathlib import Path
+
+import torch
+from safetensors import safe_open
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from eval_all import GenericThresholdCPU, get_manifest
+from cpu_programs import SUITE
+
+
+def load_tensors(path: Path):
+    out = {}
+    with safe_open(str(path), framework="pt") as f:
+        for name in f.keys():
+            out[name] = f.get_tensor(name).float()
+    return out
+
+
+def run_program(cpu: GenericThresholdCPU, mem, max_cycles: int):
+    addr_mask = (1 << cpu.addr_bits) - 1
+    state = {
+        "pc": 0,
+        "regs": [0] * 4,
+        "flags": [0] * 4,
+        "mem": list(mem),
+        "halted": False,
+        "sp": addr_mask,
+    }
+    t0 = time.perf_counter()
+    final, cycles = cpu.run(state, max_cycles=max_cycles)
+    return final, cycles, time.perf_counter() - t0
+
+
+def check_expected(final, expected: dict) -> tuple[bool, list[str]]:
+    failures = []
+    for addr, want in expected.items():
+        got = final["mem"][addr]
+        if got != want:
+            failures.append(f"M[0x{addr:04X}] = {got} (expected {want})")
+    return (len(failures) == 0), failures
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Run the CPU program suite")
+    parser.add_argument(
+        "--model", type=str,
+        default=os.path.join(os.path.dirname(__file__),
+                             "variants", "neural_computer8_small.safetensors"),
+        help="Path to .safetensors variant",
+    )
+    parser.add_argument(
+        "--only", type=str, default="",
+        help="Comma-separated subset of program names to run",
+    )
+    args = parser.parse_args()
+
+    print(f"Loading {args.model}")
+    tensors = load_tensors(Path(args.model))
+    manifest = get_manifest(tensors)
+    print(f"Manifest: data={manifest['data_bits']}-bit, addr={manifest['addr_bits']}-bit, "
+          f"mem={manifest['memory_bytes']}B")
+
+    if manifest["memory_bytes"] < 256:
+        print(f"ERROR: variant has {manifest['memory_bytes']}B memory; "
+              f"the suite needs at least 256B (scratchpad).")
+        return 2
+
+    cpu = GenericThresholdCPU(tensors)
+    only = set(s.strip() for s in args.only.split(",") if s.strip())
+
+    print()
+    print("=" * 80)
+    print(f" CPU PROGRAM SUITE  ({manifest['memory_bytes']}B mem, {manifest['data_bits']}-bit ALU)")
+    print("=" * 80)
+
+    pass_count = 0
+    fail_count = 0
+    skip_count = 0
+
+    for name, builder in SUITE:
+        if only and name not in only:
+            skip_count += 1
+            continue
+        try:
+            mem, expected, max_cycles, desc = builder(manifest["memory_bytes"])
+        except Exception as e:
+            print(f"  {name:18}  BUILD ERROR: {e}")
+            fail_count += 1
+            continue
+
+        if len(mem) != manifest["memory_bytes"]:
+            print(f"  {name:18}  SKIP (program built {len(mem)}B, "
+                  f"variant has {manifest['memory_bytes']}B)")
+            skip_count += 1
+            continue
+
+        final, cycles, elapsed = run_program(cpu, mem, max_cycles)
+        ok, failures = check_expected(final, expected)
+        if ok and not final["halted"]:
+            ok = False
+            failures.append(f"did not HALT within {max_cycles} cycles")
+
+        status = "PASS" if ok else "FAIL"
+        if ok:
+            pass_count += 1
+        else:
+            fail_count += 1
+        print(f"  {name:18}  {status}  ({cycles:>3} cyc, {elapsed:>5.2f}s)  {desc}")
+        if not ok:
+            for f in failures[:6]:
+                print(f"    - {f}")
+
+    print()
+    print("=" * 80)
+    total = pass_count + fail_count
+    print(f"  PASS: {pass_count}/{total}   FAIL: {fail_count}/{total}   SKIP: {skip_count}")
+    return 0 if fail_count == 0 else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tests/test_play.py

@@ -0,0 +1,72 @@
+"""Smoke tests for play.py.
+
+Drives play.main() against the smallest CPU variant (neural_computer8_small,
+1 KB memory) and the smallest ALU-only variant (neural_alu8). Each demo
+prints a deterministic block on stdout; we capture and grep for the success
+markers. Catches the kind of drift where play.py outpaces a refactor of the
+gate naming convention (cmp8bit, mod5, etc.) without anyone running it.
+"""
+from __future__ import annotations
+
+import io
+import os
+import sys
+import unittest
+from contextlib import redirect_stdout
+
+REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+if REPO_ROOT not in sys.path:
+    sys.path.insert(0, REPO_ROOT)
+
+import play  # noqa: E402
+
+
+def _run(model_path: str, skip_cpu: bool) -> str:
+    argv_save = sys.argv
+    sys.argv = ["play.py", "--model", model_path]
+    if skip_cpu:
+        sys.argv.append("--skip-cpu")
+    buf = io.StringIO()
+    try:
+        with redirect_stdout(buf):
+            rc = play.main()
+    finally:
+        sys.argv = argv_save
+    if rc not in (0, None):
+        raise AssertionError(f"play.main() returned {rc}")
+    return buf.getvalue()
+
+
+class TestPlay(unittest.TestCase):
+
+    def test_alu_variant(self) -> None:
+        path = os.path.join(REPO_ROOT, "variants", "neural_alu8.safetensors")
+        out = _run(path, skip_cpu=True)
+        # Booleans
+        for line in ["and      00->0 01->0 10->0 11->1",
+                     "or       00->0 01->1 10->1 11->1",
+                     "xor      00->0 01->1 10->1 11->0"]:
+            self.assertIn(line, out, f"missing boolean line: {line!r}")
+        # ALU ADD/SUB
+        self.assertIn("127 + 128 = 255 (carry=0)  expected 255  [OK]", out)
+        self.assertIn("127 - 128 = 255 (no_borrow=0)  expected 255  [OK]", out)
+        # CMP via cmp8bit cascade
+        self.assertIn("50 vs  30 -> GT=1 LT=0 EQ=0", out)
+        self.assertIn("77 vs  77 -> GT=0 LT=0 EQ=1", out)
+        self.assertIn("128 vs 127 -> GT=1 LT=0 EQ=0", out)
+        # MUL
+        self.assertIn("12 *  11 = 132  expected 132  [OK]", out)
+        # mod-5 (52 multiples of 5 in [0, 255])
+        self.assertIn("52 hits", out)
+        self.assertNotIn("FAIL", out)
+
+    def test_cpu_variant(self) -> None:
+        path = os.path.join(REPO_ROOT, "variants", "neural_computer8_small.safetensors")
+        out = _run(path, skip_cpu=False)
+        self.assertIn("M[0x0083] = 15  (expected 15)  [OK]", out)
+        self.assertIn("R0=15", out)
+        self.assertNotIn("FAIL", out)
+
+
+if __name__ == "__main__":
+    unittest.main()

todo.md

@@ -0,0 +1,21 @@
+# todo
+
+Unfinished work.
+
+- The float add, multiply, and divide pipelines handle subnormal operands and produce subnormal results rather than flushing them to zero.
+- The F extension includes a fused multiply-add instruction that rounds once.
+- FCVT.W.S honors the instruction's rounding-mode field instead of always rounding toward zero.
+- Program-counter sequencing and instruction decode are computed by threshold gates rather than by fixed wiring in the runtimes.
+- The standalone float normalize stages, superseded by the composed pipelines' own normalizers, are removed from the circuit inventory.
+- Every gate in the 32-bit multiply circuit resolves its .inputs to a defined signal.
+- A machine-checked correctness proof relates the shipped weights to the ISA.
+- The processor's full state transition is expressed as a recurrent ternary threshold network and executes as analog matrix-vector products rather than a wired gate graph.
+  - The verified gate wiring is compiled into a fixed sequence of ternary weight matrices W1..Wk with a Heaviside step between them, acting on a state vector that holds the program counter, registers, flags, and every memory bit.
+  - One instruction is that fixed stack of threshold layers, and the machine is the stack iterated until the halt bit sets, so the whole processor is a single recurrent linear-threshold network.
+  - The matrix form is proven equal to the gate-graph processor, so the ISA semantics carry over bit for bit.
+  - Each weight matrix maps to a resistive or photonic crossbar with conductances drawn from {-1, 0, +1}, and one clock cycle is one analog matrix-vector product followed by thresholding.
+  - The analog realization is shown to reproduce the digital circuit within the device's conductance-quantization and noise margins.
+- A program running on the processor emits the exact tensor file that defines that processor, and builds any other machine in the family from a description of it.
+  - The program's output stream is the byte-for-byte safetensors file of the running machine, and that equality is machine-checked rather than observed on a single run.
+  - The same program takes an encoded description of any machine in the family and constructs that machine's weight file, so the constructor is universal and the description selects what is built.
+  - Self-reproduction is the case where the description handed to the constructor is its own.

variants/neural_alu16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86cc54fedb8614bda4382d90ade7e505a29660c278f1cdf4821ac4ecbfb043e2
+size 25794970

variants/neural_alu32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89cc066ba4b2adc44af57511cfd435a9bbde16e9c13c71f91774239fbace43eb
+size 24181679

variants/neural_alu8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e339ff3e200428ccfaa933a6c07cfc81006e9dab53984708d6105923a154e6d
+size 24181679

variants/neural_computer16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6efffb459c157a77e8f98cff5eb298491f47f567b36e97aa34159a103c11bcbd
+size 34312872

variants/neural_computer16_reduced.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9c842b6270db3bc3d1a5b94a1bfcd0b0c62766aa1083cb65aed940b4984e3a8
+size 26501432

variants/neural_computer16_registers.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b26e149ccab217d877257959278bcd8cd0e262c7a9c9976a7020e38edcc0eed
+size 25882024

variants/neural_computer16_scratchpad.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecbd7f1247ae12240059e5cecbe8b9336fa197a961f0babf3154775ce3a47d97
+size 25966016

variants/neural_computer16_small.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c29e64ecab77e5c1e1809393ecdf4a0a34e4bf6a8b474b5329050d6662de0783
+size 26085300

variants/neural_computer32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb4725ab213771fa0f20c42d13a43b913cb33f1762c6a8c4b4670856f8d3592c
+size 32717681

variants/neural_computer32_reduced.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab3e572ebd2d9cbddb21fa7612cea8024131563ef873a5db9c99c9d5b9957926
+size 24906261

variants/neural_computer32_registers.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa766185ccae6f27174a12409255960b7732d9a4c9cb683a00f6a9e233bd417d
+size 24286869

variants/neural_computer32_scratchpad.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7df7be2cdea56ea2a0163276a99d8116af68d4b4eed6c5729e801b2a1c67acd7
+size 24371681

variants/neural_computer32_small.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa705c60ea36f8f54a1211a326e4d90a7e0f192ac456039e290ec5c9b9b61451
+size 24490921

variants/neural_computer8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5f5c79d0e397ca74fb22dd48804b620f57d6194d08e542205134594bd3359a3
+size 32689549

variants/neural_computer8_reduced.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ded8f1991b9f159bd917dff96e38f376c62a20ed06ab0345416b2dac63393424
+size 24878061

variants/neural_computer8_registers.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1384db2751a2031a0a9856aba10140adfb1cb91e402fe3e54a12a0af4286e8ef
+size 24258649

variants/neural_computer8_scratchpad.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:420d540595155c09be22c61900bfafd49748be87faea8ae938f4ef7b4ad3d021
+size 24343497

variants/neural_computer8_small.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8d2bbdcf4a7ca1a8bf7483f3ecacb35e45f88a75daa6cc437b2976ce34a5001
+size 24462729

variants/neural_rv32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3367ac4fd179dc05f6ed17822e327206e0ff2ab6d64c2d796448d79ca805b52b
+size 33274820

variants/neural_subleq8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3afb7184f4046421b7a6f96663227b9ef3e646389022affe8ddbb57eb52cbca5
+size 650280