deep-encoder-shallow-decoder-ko / tokenizer_config.json
snoop2head's picture
add tokenizer
be667f3
raw
history blame
81 Bytes
{"name_or_path": "korean-WPE.json", "tokenizer_class": "PreTrainedTokenizerFast"}