Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers / plots

Commit History

Upload plots/external_comparison.png with huggingface_hub
7b89ea4
verified

Ouaill commited on

Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub
e65894a
verified

Ouaill commited on

Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub
9934a1e
verified

Ouaill commited on

Upload plots/fertility_overall_trends.png with huggingface_hub
e6441c9
verified

Ouaill commited on

Upload plots/fertility_overall_comparison_v2.png with huggingface_hub
08b4403
verified

Ouaill commited on

Upload plots/fertility_overall_faceted.png with huggingface_hub
6ee779e
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_comparison_v2.png with huggingface_hub
79e19cc
verified

Ouaill commited on

Upload plots/cpt_overall_comparison_v2.png with huggingface_hub
13fbc56
verified

Ouaill commited on

Upload plots/exact_match_rate_comparison_v2.png with huggingface_hub
75e957e
verified

Ouaill commited on

Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub
5d31685
verified

Ouaill commited on

Upload plots/fertility_disparity_trends.png with huggingface_hub
83c46df
verified

Ouaill commited on

Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub
7059b0c
verified

Ouaill commited on

Upload plots/morph_consistency_f1_comparison_v2.png with huggingface_hub
bfa6f98
verified

Ouaill commited on

Upload plots/morph_consistency_f1_heatmap_v2.png with huggingface_hub
9eb984a
verified

Ouaill commited on

Upload plots/fertility_overall_comparison_v2.png with huggingface_hub
79fac54
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_heatmap_v2.png with huggingface_hub
d760658
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_trends.png with huggingface_hub
0b5bb9d
verified

Ouaill commited on

Upload plots/exact_match_rate_heatmap_v2.png with huggingface_hub
77fbb17
verified

Ouaill commited on

Upload plots/cpt_overall_trends.png with huggingface_hub
7338ed2
verified

Ouaill commited on

Upload plots/fertility_overall_heatmap_v2.png with huggingface_hub
d465e60
verified

Ouaill commited on

Upload plots/exact_match_rate_trends.png with huggingface_hub
1d04332
verified

Ouaill commited on

Upload plots/fertility_overall_trends.png with huggingface_hub
68b4b4b
verified

Ouaill commited on

Upload plots/bootstrap_ci_forest.png with huggingface_hub
7ce868c
verified

Ouaill commited on

Upload plots/external_comparison.png with huggingface_hub
0e4e973
verified

Ouaill commited on

Upload plots/morph_consistency_f1_trends.png with huggingface_hub
308a054
verified

Ouaill commited on

Upload plots/cpt_overall_faceted.png with huggingface_hub
87aaa10
verified

Ouaill commited on

Upload plots/fertility_disparity_faceted.png with huggingface_hub
b61125f
verified

Ouaill commited on

Upload plots/cpt_overall_heatmap_v2.png with huggingface_hub
7362657
verified

Ouaill commited on

Upload plots/script_comparison_v2.png with huggingface_hub
a87618b
verified

Ouaill commited on

Upload plots/oov_disparity_comparison_v2.png with huggingface_hub
2f636a1
verified

Ouaill commited on

Update comparison plot with 9 external tokenizers
1f947e5
verified

Ouaill commited on

Upload plots/external_comparison.png with huggingface_hub
02a40b6
verified

Ouaill commited on

Upload plots/script_comparison_v2.png with huggingface_hub
296f159
verified

Ouaill commited on

Upload plots/oov_disparity_comparison_v2.png with huggingface_hub
7130cb2
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_trends.png with huggingface_hub
64a1a4d
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_heatmap_v2.png with huggingface_hub
80b7629
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_comparison_v2.png with huggingface_hub
07b5369
verified

Ouaill commited on

Upload plots/morph_consistency_f1_trends.png with huggingface_hub
aeb25f3
verified

Ouaill commited on

Upload plots/morph_consistency_f1_heatmap_v2.png with huggingface_hub
af60f52
verified

Ouaill commited on

Upload plots/morph_consistency_f1_comparison_v2.png with huggingface_hub
61bf1f5
verified

Ouaill commited on

Upload plots/fertility_overall_trends.png with huggingface_hub
1bc1ca8
verified

Ouaill commited on

Upload plots/fertility_overall_heatmap_v2.png with huggingface_hub
6ee6157
verified

Ouaill commited on

Upload plots/fertility_overall_faceted.png with huggingface_hub
15905d6
verified

Ouaill commited on

Upload plots/fertility_overall_comparison_v2.png with huggingface_hub
1796080
verified

Ouaill commited on

Upload plots/fertility_disparity_trends.png with huggingface_hub
f5a9b3c
verified

Ouaill commited on

Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub
cba721e
verified

Ouaill commited on

Upload plots/fertility_disparity_faceted.png with huggingface_hub
eacdfc1
verified

Ouaill commited on

Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub
d4ef29c
verified

Ouaill commited on

Upload plots/exact_match_rate_trends.png with huggingface_hub
9e83d1e
verified

Ouaill commited on

Upload plots/exact_match_rate_heatmap_v2.png with huggingface_hub
8b1a88e
verified

Ouaill commited on