Hugging Face's logo

eljanmahammadli
/
aztext-tokenizer

Azerbaijani
tokenizer
sentencepiece
bpe
azerbaijani
low-resource
Model card Files
xet
 Community
aztext-tokenizer
2.08 MB
  • 1 contributor
 History: 2 commits
eljanmahammadli's picture
eljanmahammadli
Initial release: SentencePiece BPE 16k tokenizer for AzText
51b6b2e verified