Hugging Face's logo

flexitok
/
bpe_arb_Arab_8000

Standard Arabic
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
bpe_arb_Arab_8000

Commit History

Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub
8384471
verified

gsaltintas commited on

Upload fineweb2_hq/participating_tokenizers.json with huggingface_hub
6805476
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub
ed8cbfb
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub
89ca1c5
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub
e2ebbec
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub
265946f
verified

gsaltintas commited on

Upload folder using huggingface_hub
7508411
verified

gsaltintas commited on

Upload folder using huggingface_hub
87664a8
verified

gsaltintas commited on

initial commit
aecfd7e
verified

gsaltintas commited on