Hugging Face's logo

flexitok
/
bpe_script_Arab_16000

Standard Arabic
Persian
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_script_Arab_16000 / script_1

Commit History

Upload script_1/flexitok--bpe_script_Arab_16000_overlap.json with huggingface_hub
3c65d02
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_vocab.json with huggingface_hub
9cb8191
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_super_mapping.json with huggingface_hub
fe61b9e
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_overlap.json with huggingface_hub
4fa1ed8
verified

gsaltintas commited on

Upload script_1/participating_tokenizers.json with huggingface_hub
01f0cb3
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000.yaml with huggingface_hub
bab50ca
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_info.json with huggingface_hub
4fbdaef
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_vocab.json with huggingface_hub
13c5bbf
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_Arab_16000_super_mapping.json with huggingface_hub
0af21a6
verified

gsaltintas commited on