Hugging Face's logo

eljanmahammadli
/
aztext-tokenizer

Azerbaijani
tokenizer
sentencepiece
bpe
azerbaijani
low-resource
Model card Files
xet
 Community
aztext-tokenizer / tokenizer.model

Commit History

Initial release: SentencePiece BPE 16k tokenizer for AzText
51b6b2e
verified

eljanmahammadli commited on