agileabhi
/

hindi-bpe-tokenizer

Token Classification

Model card Files Files and versions

hindi-bpe-tokenizer

56.9 MB

1 contributor

History: 5 commits

agileabhi's picture

Updated git repo

9091999 verified about 2 months ago

.gitattributes

1.57 kB

Upload Hindi BPE Tokenizer about 2 months ago
README.md

2.31 kB

Updated git repo about 2 months ago
hindi_tokenizer.pkl
Detected Pickle imports (1)
- "hindi_tokenizer.tokenizer.HindiTokenizer"
How to fix it?
28.4 MB
xet

Upload Hindi BPE Tokenizer about 2 months ago
merges.json

78.3 kB

Upload Hindi BPE Tokenizer about 2 months ago
tokenizer_config.json

183 Bytes

Upload Hindi BPE Tokenizer about 2 months ago
vocab.json

28.5 MB
xet

Upload Hindi BPE Tokenizer about 2 months ago