Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers / results

Commit History

Upload results/external_datasets_eval.json with huggingface_hub
765c3cc
verified

Ouaill commited on

Upload results/external_datasets_eval.csv with huggingface_hub
6368cdb
verified

Ouaill commited on

Upload results/plots/dataset_comparison.png with huggingface_hub
f194887
verified

Ouaill commited on

Upload results/plots/fertility_disparity_heatmap_v2.png with huggingface_hub
c2b195c
verified

Ouaill commited on

Upload results/plots/fertility_disparity_comparison_v2.png with huggingface_hub
7e37a55
verified

Ouaill commited on

Upload results/plots/fertility_overall_trends.png with huggingface_hub
d52191a
verified

Ouaill commited on

Upload results/plots/fertility_overall_comparison_v2.png with huggingface_hub
cfa68a1
verified

Ouaill commited on

Upload results/atlaset_full_stats.json with huggingface_hub
65a552f
verified

Ouaill commited on

Upload results/dataset_stats.json with huggingface_hub
07b7716
verified

Ouaill commited on

Upload results/dataset_stats.csv with huggingface_hub
b57219a
verified

Ouaill commited on

Upload results/doda_independent_results.json with huggingface_hub
80ad519
verified

Ouaill commited on

Upload results/doda_independent_results.csv with huggingface_hub
18d35db
verified

Ouaill commited on

Upload results/codeswitch_results.json with huggingface_hub
36e7824
verified

Ouaill commited on

Upload results/codeswitch_results.csv with huggingface_hub
78cd907
verified

Ouaill commited on

Upload results/plots/external_comparison.png with huggingface_hub
7fbcaf7
verified

Ouaill commited on

Upload results/external_comparison.csv with huggingface_hub
4574143
verified

Ouaill commited on

Upload results/doda_independent_results.json with huggingface_hub
941aa6d
verified

Ouaill commited on

Upload results/doda_independent_results.csv with huggingface_hub
19ec31c
verified

Ouaill commited on

Add code-switching eval script and results
15ed237
verified

Ouaill commited on

Add code-switching eval script and results
b8d9160
verified

Ouaill commited on