Error tokenization
#2
by explorista - opened
Hey,
unfortunately I am not able to correctly load the model in my notebook. This is the error I get with all your BLEURT-20 models:
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'BleurtSPTokenizer'.
The class this function is called from is 'BertTokenizer'.
This is just a warning and can be ignored. Let me know if you encounter weird behaviors while effectively using the tokenizer.