Hugging Face's logo

flexitok
/
mod-tokenizers-rtl_2digit

Safetensors
Undetermined
llama
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
mod-tokenizers-rtl_2digit / mod_tokenizers

Commit History

Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_2digit_overlap.json with huggingface_hub
078eba3
verified

gsaltintas commited on

Upload mod_tokenizers/participating_tokenizers.json with huggingface_hub
b579c82
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_2digit.yaml with huggingface_hub
e1b180b
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_2digit_info.json with huggingface_hub
82249dc
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_2digit_vocab.json with huggingface_hub
d12cce7
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_2digit_super_mapping.json with huggingface_hub
f4e8fa5
verified

gsaltintas commited on