Shoriful025's picture
Create tokenizer_config.json
813a39a verified
// tokenizer_config.json
{
"do_lower_case": true,
"model_max_length": 512,
"padding_side": "right",
"truncation_side": "right"
}