17 days ago
-
Part 3 decoding results (finetuned)
-
Part 3: Pythia-160M fine-tuned on ROCStories
-
Part 3 decoding results (pretrained)
18 days ago
-
Upload Part 2 adamw: AdamW
-
Upload Part 1 moe_baseline: Dense 2-layer MLP (baseline)
-
Upload Part 2 sophia: Sophia-G (Hessian-based, bonus)
-
Upload Part 2 muon: Muon (matrix-based)
-
Upload Part 2 lion: Lion (memory-efficient)
-
Upload Part 2 mars: MARS (variance-reduced AdamW variant)
-
Upload Part 1 moe_equal: MoE, 4 experts, top-2, active params = dense baseline
-
Upload Part 1 moe_shared: MoE, 1 shared expert + 3 routed experts, top-1
-
Upload Part 1 moe_top2: MoE, 4 experts, top-2 routing
-
Upload Part 1 moe_top1: MoE, 4 experts, top-1 routing
-
Update index README
-
initial commit
