alstrup commited on
Commit
ee3e21e
·
verified ·
1 Parent(s): a99bce6

tier-4 SRT member: manifest.json

Browse files
Files changed (1) hide show
  1. manifest.json +3 -3
manifest.json CHANGED
@@ -2,6 +2,6 @@
2
  "entry_class": "model.ModMulRouter",
3
  "output_base": 10,
4
  "framework": "pytorch",
5
- "model_description": "Prime-magnitude router over from-random-init ByteGPT members (entmax15+ALiBi, BCE+reg): tiers 1-2 scratchpad-CoT member; tier-3 composed multiply+long-division CoT member (model generates accumulator rows, restaged dividend, per-step qd*p and remainders, answer). Routing keys on the size of p only; all answers are generated digit sequences from trained weights.",
6
- "training_description": "All members trained from random init with BCE(+reg) LM loss plus per-block BP-install probe losses on intermediate variables (carries, borrows, quotient estimates, remainder digits). Synthetic data from the official tier geometry with held-out seeds; tier-3 member trained on ~all ~6445 primes of the 9-16-bit range (full domain coverage), bit-uniform prime sampling, example-aligned batch rows. Randomising weights collapses accuracy. Training code: github.com/area9innovation/modmul-challenge (available to reviewers on request)."
7
- }
 
2
  "entry_class": "model.ModMulRouter",
3
  "output_base": 10,
4
  "framework": "pytorch",
5
+ "model_description": "Prime-magnitude router over from-random-init ByteGPT members (entmax15+ALiBi, BCE+reg): tiers 1-2 scratchpad-CoT member; tier-3 composed multiply+long-division CoT member (model generates accumulator rows, restaged dividend, per-step qd*p and remainders, answer); tier-4 SRT-division CoT member (signed quotient digits in {-9..9} from a learned leading-limbs lookup, 10's-complement remainders with emitted carry/borrow chain bits, final correction step). Routing keys on the size of p only; all answers are generated digit sequences from trained weights.",
6
+ "training_description": "All members trained from random init with BCE(+reg) LM loss plus per-block BP-install probe losses on intermediate variables (carries, borrows, quotient digits, remainder digits). Synthetic data from the official tier geometry with held-out seeds; tier-3 member trained on ~all ~6445 primes of its bit-range; tier-4 member on 50k primes of the 17-32-bit range with magnitude-balanced sampling, example-aligned batch rows. The tier-4 trace uses redundant signed-digit (SRT) long division so every per-token operation is a local lookup/chain op; the digit-selection table, products, and complement arithmetic are all generated by trained weights. Randomising weights collapses accuracy. Training code: github.com/area9innovation/modmul-challenge (available to reviewers on request)."
7
+ }