Transformers
PyTorch
text2text-generation
itazap HF Staff commited on
Commit
951418f
·
verified ·
1 Parent(s): 8c63c2b

v5 update to match TokenizersBackend to AutoTokenizer

Browse files
Files changed (1) hide show
  1. tokenizer_config.json +1 -1
tokenizer_config.json CHANGED
@@ -309,6 +309,6 @@
309
  "pad_token": "<pad>",
310
  "sp_model_kwargs": {},
311
  "spaces_between_special_tokens": false,
312
- "tokenizer_class": "T5Tokenizer",
313
  "unk_token": "<unk>"
314
  }
 
309
  "pad_token": "<pad>",
310
  "sp_model_kwargs": {},
311
  "spaces_between_special_tokens": false,
312
+ "tokenizer_class": "PreTrainedTokenizerFast",
313
  "unk_token": "<unk>"
314
  }