Hugging Face's logo

flexitok
/
bpe_ind_Latn_8000

Indonesian
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
bpe_ind_Latn_8000

Commit History

Upload fineweb2_hq/flexitok--bpe_ind_Latn_8000_overlap.json with huggingface_hub
6cf8717
verified

gsaltintas commited on

Upload fineweb2_hq/participating_tokenizers.json with huggingface_hub
3aed0a6
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_ind_Latn_8000.yaml with huggingface_hub
fac27ae
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_ind_Latn_8000_info.json with huggingface_hub
a2683c6
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_ind_Latn_8000_vocab.json with huggingface_hub
ed4be32
verified

gsaltintas commited on

Upload fineweb2_hq/flexitok--bpe_ind_Latn_8000_super_mapping.json with huggingface_hub
371ee45
verified

gsaltintas commited on

Upload folder using huggingface_hub
e412532
verified

gsaltintas commited on

Upload folder using huggingface_hub
9ceece3
verified

gsaltintas commited on

initial commit
f4bf0cf
verified

gsaltintas commited on