sajalregmi4
/

arkios-tokenizer

Model card Files Files and versions

arkios-tokenizer / tokenizer.json

Commit History

Arkios Tokenizer: 65,536-entry mark-aware Nepali-English byte-level BPE

eb795f4
verified

sajalregmi4 commited on 8 days ago