Hugging Face's logo

flexitok
/
bpe_arb_Arab_8000

Standard Arabic
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_arb_Arab_8000 / safe_v2

Commit History

Upload safe_v2/flexitok--bpe_arb_Arab_8000_overlap.json with huggingface_hub
60edd95
verified

gsaltintas commited on

Upload safe_v2/participating_tokenizers.json with huggingface_hub
22213df
verified

gsaltintas commited on

Upload safe_v2/flexitok--bpe_arb_Arab_8000.yaml with huggingface_hub
6df191d
verified

gsaltintas commited on

Upload safe_v2/flexitok--bpe_arb_Arab_8000_info.json with huggingface_hub
ea49e25
verified

gsaltintas commited on

Upload safe_v2/flexitok--bpe_arb_Arab_8000_vocab.json with huggingface_hub
a94f2f5
verified

gsaltintas commited on

Upload safe_v2/flexitok--bpe_arb_Arab_8000_super_mapping.json with huggingface_hub
927b6ea
verified

gsaltintas commited on