sajalregmi4
/

arkios-tokenizer

Model card Files Files and versions

arkios-tokenizer / tokenizer.json

sajalregmi4's picture

Arkios Tokenizer: 65,536-entry mark-aware Nepali-English byte-level BPE

eb795f4 verified 7 days ago

History Contribute Delete

5.37 MB

File too large to display, you can check the raw version instead.