Hugging Face's logo

flexitok
/
mod-tokenizers-ltr_5digit

Undetermined
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
mod-tokenizers-ltr_5digit / mod_tokenizers

Commit History

Upload mod_tokenizers/flexitok--mod-tokenizers-ltr_5digit_overlap.json with huggingface_hub
2d04b0c
verified

gsaltintas commited on

Upload mod_tokenizers/participating_tokenizers.json with huggingface_hub
4ec7ef4
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-ltr_5digit.yaml with huggingface_hub
64f933c
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-ltr_5digit_info.json with huggingface_hub
1fde34f
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-ltr_5digit_vocab.json with huggingface_hub
1fc605c
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-ltr_5digit_super_mapping.json with huggingface_hub
c06401b
verified

gsaltintas commited on