Hugging Face's logo

flexitok
/
mod-tokenizers-individual

Undetermined
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
Community
mod-tokenizers-individual

Commit History

Upload mod_tokenizers/flexitok--mod-tokenizers-individual_overlap.json with huggingface_hub
139a559
verified

gsaltintas commited on

Upload mod_tokenizers/participating_tokenizers.json with huggingface_hub
641cfa7
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-individual.yaml with huggingface_hub
2646a46
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-individual_info.json with huggingface_hub
7610dfc
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-individual_vocab.json with huggingface_hub
e049556
verified

gsaltintas commited on

Upload mod_tokenizers/flexitok--mod-tokenizers-individual_super_mapping.json with huggingface_hub
a860110
verified

gsaltintas commited on

Upload folder using huggingface_hub
1d40baf
verified

gsaltintas commited on

Upload folder using huggingface_hub
6634830
verified

gsaltintas commited on

Upload folder using huggingface_hub
0f311c6
verified

gsaltintas commited on

Upload folder using huggingface_hub
23c9c8f
verified

gsaltintas commited on

Upload folder using huggingface_hub
3d30a20
verified

gsaltintas commited on

Upload folder using huggingface_hub
ffcf824
verified

gsaltintas commited on

Upload folder using huggingface_hub
c91e831
verified

gsaltintas commited on

initial commit
77e40ab
verified

gsaltintas commited on