OiQ
/

daa-tokenizers

moroccan-darija

Model card Files Files and versions

62.5 MB

Ctrl+K

Ctrl+K

1 contributor

History: 164 commits

Ouaill's picture

Upload tokenizers/concat_az_wordpiece_16000.json with huggingface_hub

5fa57bf verified 2 months ago

plots
Upload plots/script_comparison_v2.png with huggingface_hub 2 months ago
tokenizers
Upload tokenizers/concat_az_wordpiece_16000.json with huggingface_hub 2 months ago
transformers_tokenizers
Upload transformers_tokenizers/concat_bbpe_32000_tokenizer_az/tokenizer.json with huggingface_hub 2 months ago
.gitattributes

3.29 kB
Upload plots/script_comparison_v2.png with huggingface_hub 2 months ago
README.md

11.8 kB
Upload README.md with huggingface_hub 2 months ago
benchmark_report.md

8.47 kB
Upload benchmark_report.md with huggingface_hub 2 months ago
bootstrap_ci.csv

3.23 kB
Upload bootstrap_ci.csv with huggingface_hub 2 months ago
compare_with_external.py

10.5 kB
Upload compare_with_external.py with huggingface_hub 2 months ago
eval_and_compare.py

11.5 kB
Upload eval_and_compare.py with huggingface_hub 2 months ago
external_comparison.csv

5.76 kB
Upload external_comparison.csv with huggingface_hub 2 months ago
external_comparison.json

13.2 kB
Upload external_comparison.json with huggingface_hub 2 months ago
script.py

81 kB
Upload script.py with huggingface_hub 2 months ago
tokenizer_results.csv

9.19 kB
Upload tokenizer_results.csv with huggingface_hub 2 months ago
tokenizer_results.json

20.8 kB
Upload tokenizer_results.json with huggingface_hub 2 months ago