anlp-a2-part1-moe_shared

Decoder-only transformer trained from scratch to translate Vietnamese and Japanese to English (belumind/en-vi-ja-curated-500k-triplets). FFN variant: moe_shared.

• total params: 17,018,496 | active params per token: 13,479,552
• trained on 39,003,133 tokens
• W&B run: https://wandb.ai/dunkronit-iiit-hyderabad/anlp-a2-part1/runs/moe_shared-a8f358e0

Input format: <vi> source <en> or <ja> source <en>; the model continues with English and <eos>. Load with src.part1.model.Transformer.from_pretrained("DunkRonit/anlp-a2-part1-moe_shared") from the assignment repo.