This model was pretrained using the MosaicML Composer framework. Tokenizer used: modernBERT-base Initialised from latest stable checkpoint of modernBERT-base model: checkpoints/context-extension/ep0-ba52988-rank0.pt
Files info