almaghrabima
/

SARFTokenizer

Text Generation

Model card Files Files and versions

Commit History

Delete morfessor_models

f2c4997
verified

almaghrabima commited on Feb 14

Delete lexicons

16e88db
verified

almaghrabima commited on Feb 14

Upload lexicons/lexicon_en.txt with huggingface_hub

eef9b5a
verified

almaghrabima commited on Feb 8

Upload lexicons/lexicon_ar.txt with huggingface_hub

453d9e0
verified

almaghrabima commited on Feb 8

Upload morfessor_models/morf_map_reverse.json with huggingface_hub

2772bff
verified

almaghrabima commited on Feb 8

Upload morfessor_models/morf_map.json with huggingface_hub

5fee9e3
verified

almaghrabima commited on Feb 8

Upload morfessor_models/morfessor_en.bin with huggingface_hub

77f39b7
verified

almaghrabima commited on Feb 8

Upload morfessor_models/morfessor_ar.bin with huggingface_hub

7309795
verified

almaghrabima commited on Feb 8

Update README.md

c94c9a8
verified

almaghrabima commited on Feb 7

Upload benchmark_pypi_full.py with huggingface_hub

4e2b74e
verified

almaghrabima commited on Feb 5

Delete benchmark_pypi.py with huggingface_hub

e86ee8e
verified

almaghrabima commited on Feb 5

Upload tokenizer_config.json with huggingface_hub

dfd4850
verified

almaghrabima commited on Feb 5

Upload special_tokens_map.json with huggingface_hub

f6b77da
verified

almaghrabima commited on Feb 5

Upload tokenizer.json with huggingface_hub

a5801cf
verified

almaghrabima commited on Feb 5

Update README.md

344b20c
verified

almaghrabima commited on Feb 5

Update README.md

2d1f910
verified

almaghrabima commited on Feb 5

Update README.md

d3c23ec
verified

almaghrabima commited on Feb 5

Update benchmark results with new tokenizers (Falcon-H1, ALLaM, Hala, Mistral)

55db6a1
verified

almaghrabima commited on Feb 4

Upload benchmark_parallel_results.json with huggingface_hub

a88f8a7
verified

almaghrabima commited on Feb 4

Upload test_comprehensive_results.json with huggingface_hub

1e8911f
verified

almaghrabima commited on Feb 4

Upload BENCHMARK_REPORT.md with huggingface_hub

57f81bb
verified

almaghrabima commited on Feb 4

Upload test_comprehensive_million.py with huggingface_hub

c24518d
verified

almaghrabima commited on Feb 4

Upload benchmark_tiktoken_style.py with huggingface_hub

9770614
verified

almaghrabima commited on Feb 4

Upload README.md with huggingface_hub

e180732
verified

almaghrabima commited on Feb 4

Upload README.md with huggingface_hub

37ba042
verified

almaghrabima commited on Feb 4

Upload README.md with huggingface_hub

6aa17b2
verified

almaghrabima commited on Feb 4

Upload requirements.txt with huggingface_hub

edaafcb
verified

almaghrabima commited on Feb 4

Upload benchmark_pypi.py with huggingface_hub

169c0b4
verified

almaghrabima commited on Feb 4

Upload README.md with huggingface_hub

ce0f0a1
verified

almaghrabima commited on Feb 4

Upload morf_map.bin with huggingface_hub

c90e6b5
verified

almaghrabima commited on Feb 4

Upload tokenizer.json with huggingface_hub

183d871
verified

almaghrabima commited on Feb 4

initial commit

213dca4
verified

almaghrabima commited on Feb 4