th4kur/anlp-a2 — ANLP Assignment 2 checkpoints

One repository, one folder per model. Every folder holds model.safetensors, config.json, tokenizer.json, metrics.json and its own README.md.

Part 1 — FFN variants for vi/ja -> en translation (part1/<variant>/)

folder	params	test ppl	test BLEU
part1/dense	27,269,632	3.94	38.03
part1/moe_top1	27,281,920	4.466	35.89
part1/moe_top2	27,281,920	4.054	37.44
part1/moe_shared	27,278,848	4.089	37.34
part1/moe_top2_iso_active	39,864,832	3.865	38.42

Part 2 — optimizers, pre-training on HAP-E (part2/<optimizer>/)

folder	params	val loss	test BLEU
part2/adamw	27,269,632	3.845	3.32
part2/mars	27,269,632	3.836	3.335
part2/lion	27,269,632	3.743	3.772
part2/muon	27,269,632	3.809	3.786
part2/sophia	27,269,632	3.802	3.504

Part 3 — decoding strategies on Pythia-160M (part3/pythia-160m/)

The unmodified pretrained EleutherAI/pythia-160m (162,322,944 parameters), byte-identical to the original; nothing was trained. The decoding code lives in the assignment's git repository, not here. See the folder's own README for the results table.

Loading

from src.part1.hub import load_model   # from the assignment code
model, cfg = load_model("th4kur/anlp-a2", subfolder="part1/dense")