splade_morpheme / tokenizer_config.json
HeyDunaX's picture
upload tokenizer_config.json
a5d98a6 verified
Raw
History Blame Contribute Delete
234 Bytes
{
"tokenizer_class": "MorphemeTokenizer",
"model_type": "splade",
"vocab_size": 57030,
"max_length": 128,
"do_lower_case": false,
"seg_temp_train": 1.0,
"seg_temp_eval": 0.0,
"min_freq_word": 2,
"min_freq_morph": 3
}