janakhpon
/

mon_tokenizer

Model card Files Files and versions

mon_tokenizer / model_card.json

Commit History

fix: correct the single-token coverage claim to 98.74%

b2eb67d

janakhpon commited on 6 days ago

feat: migrate the artifact from SentencePiece to tokenizers JSON

954bd9d

janakhpon commited on 11 days ago