| "model_description": "Bit-sequential RNN (~10.7M params across two shared carry-aware TCN weight-sets) for modular multiplication with primes up to 2^2048. The model reads the bits of a mod p MSB-first, one per step, conditioned on (b mod p, p) in binary. Its hidden state is a hard-quantized bit vector, and the transition function is a learned carry-aware dilated-convolution TCN trained to implement the Horner step (t, bit, b, p) -> (2*t + bit*b) mod p. The final hidden state bits are emitted MSB-first as the base-2 answer. Routing uses the narrowest state width that can hold p. A shared TCN weight-set, weights_shared_16_512.pt, serves 16/32/64/128/256/512-bit states (tiers 1-8) at each prime's native width and reaches tiers 1-8 = 1.00 on the public benchmark. A second shared TCN weight-set, weights_shared_1024_2048.pt, serves 1024/2048-bit states (tiers 9-10) at native width and reaches tiers 9 and 10 = 1.00 on the public benchmark. For p >= 2^2048 the model emits the honest [0] fallback without invoking the network. The arithmetic is not in Python code: tokenization, scan, threshold, and readout are architecture, while doubling, conditional add, compare/borrow, and reduction are all learned in the trained cell weights; random or perturbed weights collapse to the floor.", |
| "training_description": "Each cell is trained from exact single-step labels (t, bit, b, p) -> (2*t + bit*b) mod p, with BCE per state bit, AdamW, cosine decay, gradient clipping, EMA checkpointing, and held-out-prime validation. Training data uses true Horner-trajectory states plus boundary-focused examples; prime sampling is value-uniform to match the challenge generator, with bit-length-uniform bands where needed so the reduction boundary is seen at every position. The shared 16-512 file was built by warm-starting from the shipped shared 64-512 carry-aware TCN (14 residual TCN blocks, 256 channels, dilations cycling through 1..256), fine-tuning on a {16,32,64,128,256,512} width mix, then averaging that run with a small-tier polish tail: soup25 = 0.75 * unified_16to512_warm_s0.final + 0.25 * unified_16to512_smalltail_s1.final. On the fixed public benchmark this merge brought the small/mid tiers 1-8 to 1.00. A matched faithful bootstrap over tiers 1-8 (5 primes/tier structure, pool120, k30, boot200k, seed 515151) ties tiers 1-2 and improves tiers 3-8; tier 8 E[acc] improves 0.9866 -> 0.9931 and P(tier8<0.95) drops 1.396% -> 0.205%. The shared 1024-2048 file was built by warm-starting from the public-correct 2048-bit TCN (13 blocks, max_dil 1024) and training jointly at widths 1024 and 2048 with logit-distillation to BOTH dedicated teachers (the 1024-width logits toward the strong dedicated 1024 cell, which transfers its 1024 chain robustness; the 2048-width logits toward the dedicated 2048 cell, which holds the tier-10 primary key) plus a worst-bit margin loss, under a 2048 chain-preservation floor so no tier-10-eroding checkpoint can be saved. This makes one shared cell match both dedicated cells at their own widths without any model-soup. An earlier soup route (0.70 * old weights2048 + 0.30 * a 1024/2048 pilot) held tier 10 but regressed tier 9 under a faithful 5-prime bootstrap (E 0.968, worst-prime 0.80, because the old 2048 cell is only ~0.94 at native 1024), so it was dropped. Faithful gate (diag_5prime_boot, pool 100, seed 991): tier 9 E[acc] 0.9939 / worst-prime 0.933 (matching the dedicated 1024 cell), tier 10 E[acc] 0.9913 / P(acc<0.90) 0.002% / worst-prime 0.933 (primary key held). Public benchmark: overall_accuracy 1.00, tiers 1-10 all 1.00, highest_tier_above_90 = 10, deterministic. Full all-width single-cell unification across 16..2048 was tested and rejected because one ~5M cell could not preserve 2048-chain robustness while serving small/mid widths; the shipped design intentionally keeps two adjacent shared groups. Compliance checks: preprocess hooks are identity, the legal two-operand reductions a%p and b%p are used only for input normalization, perturbing trained weights collapses accuracy toward the untrained floor, and held-out-prime generalization tracks train accuracy." |
