gwlms
/

wordpiece-tokenizer

Model card Files Files and versions

wordpiece-tokenizer / tokenizer_config.json

stefan-it's picture

tokenizer: add initial config and vocab

2291e32 over 2 years ago

history blame contribute delete

60 Bytes

	{"do_lower_case": false, "max_len": 512, "init_inputs": []}