Fixed bug tokenizer SecureBERT
#1
by terranovaa - opened
Fixed bug fast tokenizer, to be disabled for SecureBERT
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sana9 changed pull request status to merged