c3-ft110m-v4

Fine-tuned probe model for the C3 project (calibration of diffusion language models): the 110M MDLM from the v4 training run (ANLP-project/c3-mdlm-110m-v4 lineage; architecture 12 layers, d_model 768, 12 heads, vocab 50258, max length 256) fine-tuned for ~100M tokens (18,393 steps) with the MDLM objective, window_mode: prefix, lr 1e-4, warmup 300, EMA 0.999, seed 20260917, config hash 9057b964025af1fd. Full config in train_summary.json; loss curves in history.jsonl.

  • Training data: probe shards derived from the project's FineWeb-Edu pipeline (ODC-By 1.0, https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu; Penedo et al., 2024). The data is not redistributed here.
  • Files: final.pt (PyTorch pickle, not converted to safetensors), history.jsonl, train_summary.json.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train ANLP-project/c3-ft110m-v4