base2 / tokenizer_config.json
gszabo's picture
Upload tokenizer
4a830b4
{
"tokenizer_class": "PreTrainedTokenizerFast"
}