Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers

Commit History

Upload results/external_datasets_eval.json with huggingface_hub
765c3cc
verified

Ouaill commited on

Upload results/external_datasets_eval.csv with huggingface_hub
6368cdb
verified

Ouaill commited on

Upload eval_external_datasets.py with huggingface_hub
7ce8a98
verified

Ouaill commited on

Upload gen_dataset_figure.py with huggingface_hub
24af0fa
verified

Ouaill commited on

Upload results/plots/dataset_comparison.png with huggingface_hub
f194887
verified

Ouaill commited on

Upload results/plots/fertility_disparity_heatmap_v2.png with huggingface_hub
c2b195c
verified

Ouaill commited on

Upload results/plots/fertility_disparity_comparison_v2.png with huggingface_hub
7e37a55
verified

Ouaill commited on

Upload results/plots/fertility_overall_trends.png with huggingface_hub
d52191a
verified

Ouaill commited on

Upload results/plots/fertility_overall_comparison_v2.png with huggingface_hub
cfa68a1
verified

Ouaill commited on

Upload figures/dataset_comparison.png with huggingface_hub
5153c8f
verified

Ouaill commited on

Upload figures/fertility_disparity_heatmap_v2.png with huggingface_hub
70e953c
verified

Ouaill commited on

Upload figures/fertility_disparity_comparison_v2.png with huggingface_hub
80c0af0
verified

Ouaill commited on

Upload figures/fertility_overall_trends.png with huggingface_hub
d7733e0
verified

Ouaill commited on

Upload figures/fertility_overall_comparison_v2.png with huggingface_hub
04cd003
verified

Ouaill commited on

Upload results/atlaset_full_stats.json with huggingface_hub
65a552f
verified

Ouaill commited on

Upload results/dataset_stats.json with huggingface_hub
07b7716
verified

Ouaill commited on

Upload results/dataset_stats.csv with huggingface_hub
b57219a
verified

Ouaill commited on

Upload dataset_stats.py with huggingface_hub
ec4201a
verified

Ouaill commited on

Upload eval_doda_independent.py with huggingface_hub
6d55a8f
verified

Ouaill commited on

Upload eval_codeswitch_and_new_baselines.py with huggingface_hub
d31cccc
verified

Ouaill commited on

Upload results/doda_independent_results.json with huggingface_hub
80ad519
verified

Ouaill commited on

Upload results/doda_independent_results.csv with huggingface_hub
18d35db
verified

Ouaill commited on

Upload results/codeswitch_results.json with huggingface_hub
36e7824
verified

Ouaill commited on

Upload results/codeswitch_results.csv with huggingface_hub
78cd907
verified

Ouaill commited on

Upload eval_darijabert_mix.py with huggingface_hub
0cadf33
verified

Ouaill commited on

Upload results/plots/external_comparison.png with huggingface_hub
7fbcaf7
verified

Ouaill commited on

Upload results/external_comparison.csv with huggingface_hub
4574143
verified

Ouaill commited on

Upload figures/external_comparison.png with huggingface_hub
061d277
verified

Ouaill commited on

Upload latest_main.pdf with huggingface_hub
c9a1889
verified

Ouaill commited on

Upload latest_main.tex with huggingface_hub
584d5e6
verified

Ouaill commited on

Upload README.md with huggingface_hub
b7fbcec
verified

Ouaill commited on

Upload latest_main.pdf with huggingface_hub
3499545
verified

Ouaill commited on

Upload latest_main.tex with huggingface_hub
0b95151
verified

Ouaill commited on

Upload README.md with huggingface_hub
0687ce2
verified

Ouaill commited on

Upload latest_main.tex with huggingface_hub
153f03d
verified

Ouaill commited on

Upload eval_morph_large.py with huggingface_hub
fc0528a
verified

Ouaill commited on

Upload morph_large_vocab_results.csv with huggingface_hub
8025e6f
verified

Ouaill commited on

Upload code.md with huggingface_hub
1ae8a3c
verified

Ouaill commited on

Upload README.md with huggingface_hub
29cabec
verified

Ouaill commited on

Upload plots/external_comparison.png with huggingface_hub
7b89ea4
verified

Ouaill commited on

Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub
e65894a
verified

Ouaill commited on

Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub
9934a1e
verified

Ouaill commited on

Upload plots/fertility_overall_trends.png with huggingface_hub
e6441c9
verified

Ouaill commited on

Upload plots/fertility_overall_comparison_v2.png with huggingface_hub
08b4403
verified

Ouaill commited on

Upload figures/external_comparison.png with huggingface_hub
8289834
verified

Ouaill commited on

Upload figures/fertility_disparity_heatmap_v2.png with huggingface_hub
8f847de
verified

Ouaill commited on

Upload figures/fertility_disparity_comparison_v2.png with huggingface_hub
2bc3559
verified

Ouaill commited on

Upload figures/fertility_overall_trends.png with huggingface_hub
2ef0028
verified

Ouaill commited on

Upload figures/fertility_overall_comparison_v2.png with huggingface_hub
f4de84a
verified

Ouaill commited on

Upload latest_main.tex with huggingface_hub
28cfed3
verified

Ouaill commited on