beerttt / tokenizer

Commit History

Delete tokenizer/special_tokens_map
f1b5d9b
verified

subbunanepalli commited on

Rename special_tokens_map to tokenizer/special_tokens_map
73cd28e
verified

subbunanepalli commited on

Create tokenizer/tokenizer_config
c76fa6a
verified

subbunanepalli commited on