bitserial-modmul-v3 / manifest.json
TrickyRex's picture
bit-serial learned reducer
e8d13e3 verified
Raw
History Blame Contribute Delete
2.01 kB
{
"entry_class": "model.BitSerialReducer",
"output_base": 2,
"framework": "pytorch",
"model_description": "One shared, p-conditioned recurrent transition cell (~471K parameters: a bidirectional 2-layer GRU over three bit-channels, a control-bit embedding, a per-bit output head) applied in a fixed bit-serial Horner loop, at 128-bit state width. Each operand is tokenised per-argument into its MSB-first bit list; the modulus is fed as its 128-bit binary form (extracted via 32-bit limbs so values above 2^63 do not overflow). The cell maps (state_bits, multiplicand_bits, modulus_bits, control_bit) to the next state bits and is used with shared weights to reduce a mod p, reduce b mod p (multiplicand 1), and multiply the two residues (multiplicand a mod p, control bits scanning b mod p). Answers are emitted as base-2 digits and reconstructed by the harness decoder. State is carried as bits between steps; no integer reconstruction or modular product happens in Python. Trained regime is primes below 2^128 and operands up to 512 bits (tiers 1-6); outside it the model abstains and emits a single zero. Same architecture as bit-serial-v1/v2, widened to 128 bits.",
"training_description": "Trained from random initialisation on one-step transitions s' = (2*s + d*x) mod p sampled over moduli covering tiers 1-6 (modulus bit-length stratified across 1-128 bits, wrap-boundary transitions oversampled). Objective is per-output-bit binary cross-entropy; optimiser AdamW (peak lr 1.5e-3, weight decay 0.01, gradient clipping) with lr warmup and cosine decay, 50k steps of batch 256 on an NVIDIA A100. No precomputed tables, no hand-coded reduction or multiplication: the per-step modular transition (including the conditional wrap, at most two subtractions of p since 2*s + d*x < 3*p) is what is learned; the loop only sequences the bits. The capability lives in the weights, so randomising them collapses exact-match accuracy to chance. Evaluation primes are unseen during training (secret seed)."
}