Q-KGR / transformers-4.33.3 /utils /test_module /custom_tokenization.py
271754echo's picture
Upload folder using huggingface_hub
4844162 verified
from transformers import BertTokenizer
class CustomTokenizer(BertTokenizer):
pass