Hugging Face's logo

eljanmahammadli
/
aztext-tokenizer

Azerbaijani
tokenizer
sentencepiece
bpe
azerbaijani
low-resource
Model card Files
xet
 Community
aztext-tokenizer / tokenizer.json
eljanmahammadli's picture
eljanmahammadli
Initial release: SentencePiece BPE 16k tokenizer for AzText
51b6b2e verified
raw
history contribute delete
1.58 MB
File too large to display, you can check the raw version instead.