thedeba
/

deb-bpe

Model card Files Files and versions

deb-bpe / tokenizer.json

thedeba's picture

Add bilingual Bengali-English BPE tokenizer (100k vocab, byte_fallback)

9a0fd03 verified about 1 month ago

History Contribute Delete

4.36 MB

File too large to display, you can check the raw version instead.