Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers / tokenizers

Commit History

Upload tokenizers/concat_az_bbpe_55000.json with huggingface_hub
6271d4a
verified

Ouaill commited on

Upload tokenizers/concat_ar_bbpe_55000.json with huggingface_hub
b0d9588
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_55000.json with huggingface_hub
473b885
verified

Ouaill commited on

Upload tokenizers/concat_ar_wordpiece_55000.json with huggingface_hub
45b9fbf
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_55000.json with huggingface_hub
d6074e5
verified

Ouaill commited on

Upload tokenizers/concat_ar_unigram_55000.json with huggingface_hub
be8d5ce
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_55000.json with huggingface_hub
c844c8b
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_55000.json with huggingface_hub
8f9aec4
verified

Ouaill commited on

Upload tokenizers/concat_az_bbpe_40000.json with huggingface_hub
c91806d
verified

Ouaill commited on

Upload tokenizers/concat_ar_bbpe_40000.json with huggingface_hub
c13d967
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_40000.json with huggingface_hub
ceddd9d
verified

Ouaill commited on

Upload tokenizers/concat_ar_wordpiece_40000.json with huggingface_hub
317fc7b
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_40000.json with huggingface_hub
bb54448
verified

Ouaill commited on

Upload tokenizers/concat_ar_unigram_40000.json with huggingface_hub
575deaf
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_40000.json with huggingface_hub
c76c893
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_40000.json with huggingface_hub
a1760cc
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_110000.json with huggingface_hub
b52a919
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_110000.json with huggingface_hub
a3a9d89
verified

Ouaill commited on

Upload tokenizers/shared_unigram_110000.json with huggingface_hub
52db63a
verified

Ouaill commited on

Upload tokenizers/shared_bpe_110000.json with huggingface_hub
458c81b
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_80000.json with huggingface_hub
9d726e7
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_80000.json with huggingface_hub
d467a27
verified

Ouaill commited on

Upload tokenizers/shared_unigram_80000.json with huggingface_hub
b586f2b
verified

Ouaill commited on

Upload tokenizers/shared_bpe_80000.json with huggingface_hub
81a1702
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_16000.json with huggingface_hub
a2b3e8d
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_16000.json with huggingface_hub
476c81a
verified

Ouaill commited on

Upload tokenizers/concat_az_bbpe_16000.json with huggingface_hub
511b189
verified

Ouaill commited on

Upload tokenizers/concat_ar_bbpe_16000.json with huggingface_hub
7e91feb
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_16000.json with huggingface_hub
5fa57bf
verified

Ouaill commited on

Upload tokenizers/concat_ar_wordpiece_16000.json with huggingface_hub
ea520a8
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_16000.json with huggingface_hub
36cc966
verified

Ouaill commited on

Upload tokenizers/concat_ar_unigram_16000.json with huggingface_hub
7515aab
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_16000.json with huggingface_hub
d97579f
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_16000.json with huggingface_hub
0797ad4
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_8000.json with huggingface_hub
4e94bfd
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_32000.json with huggingface_hub
3bad3ed
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_16000.json with huggingface_hub
5bcad97
verified

Ouaill commited on

Upload tokenizers/shared_unigram_8000.json with huggingface_hub
89541e2
verified

Ouaill commited on

Upload tokenizers/shared_unigram_32000.json with huggingface_hub
77a34c6
verified

Ouaill commited on

Upload tokenizers/shared_unigram_16000.json with huggingface_hub
900ab6f
verified

Ouaill commited on

Upload tokenizers/shared_bpe_8000.json with huggingface_hub
64de7d4
verified

Ouaill commited on

Upload tokenizers/shared_bpe_32000.json with huggingface_hub
af98261
verified

Ouaill commited on

Upload tokenizers/shared_bpe_16000.json with huggingface_hub
f55bb08
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_8000.json with huggingface_hub
bde3e36
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_32000.json with huggingface_hub
3b09fbe
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_16000.json with huggingface_hub
9ecdfa8
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_8000.json with huggingface_hub
e364e93
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_4000.json with huggingface_hub
aadde51
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_16000.json with huggingface_hub
1120a3a
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_8000.json with huggingface_hub
1391d68
verified

Ouaill commited on