almaghrabima
/

deeplatent-tokenizer

Model card Files Files and versions

deeplatent-tokenizer

9.24 MB

1 contributor

History: 28 commits

almaghrabima's picture

Update README: remove morpheme_map.json references (now bundled in suhail-nlp)

a28373d verified 8 days ago

.gitattributes
127 Bytes

Upload .gitattributes with huggingface_hub 9 days ago
README.md
6.3 kB

Update README: remove morpheme_map.json references (now bundled in suhail-nlp) 8 days ago
special_tokens_map.json
449 Bytes

Upload special_tokens_map.json with huggingface_hub 9 days ago
token_bytes.pt
402 kB
xet

Upload token_bytes.pt with huggingface_hub 9 days ago
tokenizer.json
7.54 MB

Upload tokenizer.json with huggingface_hub 9 days ago
tokenizer.pkl
1.29 MB
xet

Upload tokenizer.pkl with huggingface_hub 9 days ago
tokenizer_config.json
631 Bytes

Upload tokenizer_config.json with huggingface_hub 9 days ago