alstrup commited on
Commit
808882d
·
verified ·
1 Parent(s): 617b960

cleanup submission: manifest.json

Browse files
Files changed (1) hide show
  1. manifest.json +3 -3
manifest.json CHANGED
@@ -2,6 +2,6 @@
2
  "entry_class": "model.ModMulRouter",
3
  "output_base": 10,
4
  "framework": "pytorch",
5
- "model_description": "Prime-magnitude router over from-random-init ByteGPT members (entmax15+ALiBi, BCE+reg): tiers 1-2 scratchpad-CoT member; tier-3 composed multiply+long-division CoT member (model generates accumulator rows, restaged dividend, per-step qd*p and remainders, answer); tier-4 SRT-division CoT member (signed quotient digits in {-9..9} from a learned leading-limbs lookup, 10's-complement remainders with emitted carry/borrow chain bits, final correction step). Routing keys on the size of p only; all answers are generated digit sequences from trained weights.",
6
- "training_description": "All members trained from random init with BCE(+reg) LM loss plus per-block BP-install probe losses on intermediate variables (carries, borrows, quotient digits, remainder digits). Synthetic data from the official tier geometry with held-out seeds; tier-3 member trained on ~all ~6445 primes of its bit-range; tier-4 member on 50k primes of the 17-32-bit range with magnitude-balanced sampling, example-aligned batch rows. The tier-4 trace uses redundant signed-digit (SRT) long division so every per-token operation is a local lookup/chain op; the digit-selection table, products, and complement arithmetic are all generated by trained weights. Randomising weights collapses accuracy. Training code: github.com/area9innovation/modmul-challenge (available to reviewers on request)."
7
- }
 
2
  "entry_class": "model.ModMulRouter",
3
  "output_base": 10,
4
  "framework": "pytorch",
5
+ "model_description": "Prime-magnitude router over four from-random-init ByteGPT members (entmax15+ALiBi, BCE+reg). Tiers 1-4 members generate SRT-style signed-digit long-division chains of thought (signed quotient digits in {-9..9} from a learned leading-limbs lookup, 10's-complement remainders with emitted carry/borrow chain bits, final correction step) after a schoolbook multiply stage; a multiplication-only member covers larger p whose products fit 20-digit operands (tier-0 style inputs; oversized operands emit the honest [0]). Routing keys on the size of p only; all answers are generated digit sequences from trained weights.",
6
+ "training_description": "All members trained from random init with BCE(+reg) LM loss plus per-block BP-install probe losses on intermediate variables (carries, borrows, quotient digits). Synthetic data from the official tier geometry with held-out seeds; magnitude-balanced prime sampling; example-aligned batch rows. SRT members additionally train on alternate-valid-digit (dithered) traces so generation is robust to either choice in the redundant digit windows. The digit-selection tables, products, and complement arithmetic are all generated by trained weights; randomising weights collapses accuracy. Training code: github.com/area9innovation/modmul-challenge (available to organizers/reviewers on request)."
7
+ }