HuggingFaceFW/fineweb-edu
Viewer โข Updated โข 3.5B โข 437k โข 1.39k
Fine-tuned probe model for the C3 project (calibration of diffusion language models): the 110M MDLM from the v4 training run (ANLP-project/c3-mdlm-110m-v4 lineage; architecture 12 layers, d_model 768, 12 heads, vocab 50258, max length 256) fine-tuned for ~100M tokens (18,393 steps) with the MDLM objective, window_mode: prefix, lr 1e-4, warmup 300, EMA 0.999, seed 20260917, config hash 9057b964025af1fd. Full config in train_summary.json; loss curves in history.jsonl.
final.pt (PyTorch pickle, not converted to safetensors), history.jsonl, train_summary.json.