mysent / tokenizer_config.json
chrmongeau's picture
Update tokenizer_config.json
b810339 verified
raw
history blame contribute delete
84 Bytes
{"do_lower_case": true, "model_max_length": 512, "tokenizer_class": "BertTokenizer"}