Hugging Face's logo

tampakwill
/
AWA-150M-Tokenizer-V1

Transformers
Model card Files
xet
 Community
AWA-150M-Tokenizer-V1

Commit History

Upgrade to 8k Vocab - Support Al-Qur'an & Indo (No More Broken Bytes)
f82d0b5
verified

tampakwill commited on

Upgrade to 50k Vocab - Support Al-Qur'an & Indo (No More Broken Bytes)
568fb9d
verified

tampakwill commited on

Upload tokenizer
bf15648
verified

tampakwill commited on

Upload tokenizer
93c0e4b
verified

tampakwill commited on

Upload tokenizer
ca5d50c
verified

tampakwill commited on

initial commit
49a8a05
verified

tampakwill commited on