Hugging Face's logo

flexitok
/
bpe_arb_Arab_8000

Standard Arabic
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_arb_Arab_8000 / safe

Commit History

Upload safe/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub
96fdf3c
verified

gsaltintas commited on

Upload safe/participating_tokenizers.json with huggingface_hub
051100a
verified

gsaltintas commited on

Upload safe/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub
2f6e2c8
verified

gsaltintas commited on

Upload safe/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub
a6ede60
verified

gsaltintas commited on

Upload safe/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub
5e78f80
verified

gsaltintas commited on

Upload safe/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub
8e14c98
verified

gsaltintas commited on