flexitok/unigram_dan_Latn_8000
Updated
Edit Models filters
Apps
Inference Providers
Active filters: tokenizer
flexitok/unigram_tur_Latn_8000
Updated
flexitok/bpe_vie_Latn_8000
Updated
flexitok/unigram_hun_Latn_8000
Updated
flexitok/unigram_por_Latn_8000
Updated
flexitok/bpe_fra_Latn_8000
Updated
flexitok/bpe_spa_Latn_8000
Updated
flexitok/unigram_ind_Latn_8000
Updated
flexitok/bpe_swe_Latn_8000
Updated
flexitok/unigram_fas_Arab_8000
Updated
flexitok/bpe_deu_Latn_8000
Updated
flexitok/bpe_rus_Cyrl_8000
Updated
flexitok/bpe_fw_edu_8000
Updated
flexitok/bpe_jpn_Jpan_8000
Updated
flexitok/mod-tokenizers-bpe_numeric_268
Updated
flexitok/mod-tokenizers-bpe_numeric_1
Updated
flexitok/bpe_cmn_Hani_8000
Updated
kalixlouiis/myX-Tokenizer
Feature Extraction • Updated
DatarrX/myX-Tokenizer
Feature Extraction • Updated
flexitok/mod-tokenizers-individual
3.42M • Updated • 36
flexitok/mod-tokenizers-ltr_3digit
3.93M • Updated • 79
flexitok/mod-tokenizers-ltr_2digit
3.46M • Updated • 72
flexitok/mod-tokenizers-ltr_4digit
8.53M • Updated • 51
flexitok/mod-tokenizers-ltr_5digit
54.6M • Updated • 25
flexitok/mod-tokenizers-rtl_2digit
3.46M • Updated • 71
flexitok/mod-tokenizers-rtl_3digit
3.93M • Updated • 73
flexitok/mod-tokenizers-rtl_4digit
8.53M • Updated • 66
flexitok/mod-tokenizers-rtl_5digit
54.6M • Updated • 29
syhrlhyn/aibys-tokenizer
Updated
alibayram/magibu-3.5-128k-processor
Updated