hf-internal-testing
/

test_dynamic_tokenizer_legacy

Model card Files Files and versions

test_dynamic_tokenizer_legacy / tokenization.py

sgugger's picture

Initial commit

2ebb1d0 almost 4 years ago

history blame contribute delete

112 Bytes

	from transformers import BertTokenizer


	class NewTokenizer(BertTokenizer):
	special_attribute_present =True