mgpt2-sft / tokenization_mgpt2.py
ace-1's picture
Publish mgpt2 sft checkpoint (step 1262, val_loss 1.240358)
09246b1 verified
raw
history blame contribute delete
80 Bytes
from tokenizer.hf_tokenizer import MGPT2Tokenizer
__all__ = ['MGPT2Tokenizer']