Marin-8B plain CPT control, mid-trained @ 4e-4
Plain continued-pretraining control for the MHAR mid-training pair:
marin-8b-base (revision phoenix) mid-trained on anneal_pt_v3 for 9,500 steps
(lr 4e-4 -> 4e-6, ~10B tokens, global batch ~1.05M tokens), identical
schedule/data order as wdlctc/marin-8b-cpt-mhar-delta. Final EMA
checkpoint. Standard Llama architecture.
Eval (final EMA): GSM8K 47.0 (strict), GPQA 31.5, MMLU 64.3, HumanEval 40.9, MBPP 38.6, MATH 19.1 (math_verify).
- Downloads last month
- 12
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for wdlctc/marin-8b-cpt-plain-lr4e4
Base model
marin-community/marin-8b-base