flexitok
/

mod-tokenizers-rtl_4digit

Model card Files Files and versions

mod-tokenizers-rtl_4digit

903 kB

Ctrl+K

Ctrl+K

1 contributor

History: 24 commits

gsaltintas's picture

Upload tokenizer file participating_tokenizers.json - Upload model files

92827ac verified about 2 months ago

mod_tokenizers
Upload mod_tokenizers/flexitok--mod-tokenizers-rtl_4digit_overlap.json with huggingface_hub about 2 months ago
.gitattributes

1.52 kB
initial commit about 2 months ago
README.md

1.1 kB
Upload folder using huggingface_hub about 2 months ago
flexitok--mod-tokenizers-rtl_4digit_overlap.json

586 Bytes
Upload tokenizer file flexitok--mod-tokenizers-rtl_4digit_overlap.json - Upload model files about 2 months ago
flexitok--mod-tokenizers-rtl_4digit_super_mapping.json

148 kB
Upload tokenizer file flexitok--mod-tokenizers-rtl_4digit_super_mapping.json - Upload model files about 2 months ago
merges.txt

129 kB
Upload folder using huggingface_hub about 2 months ago
participating_tokenizers.json

342 Bytes
Upload tokenizer file participating_tokenizers.json - Upload model files about 2 months ago
special_tokens_map.json

162 Bytes
Upload folder using huggingface_hub about 2 months ago
tokenizer.json

158 kB
Upload tokenizer file flexitok--mod-tokenizers-rtl_4digit_vocab.json - Upload model files about 2 months ago
tokenizer_config.json

94 Bytes
Upload tokenizer file flexitok--mod-tokenizers-rtl_4digit_info.json - Upload model files about 2 months ago
vocab.json

158 kB
Upload folder using huggingface_hub about 2 months ago