example / tokenizer_config.json

Commit History

Fix tokenizer_config: move lang codes to additional_special_tokens for transformers<5 compat
a64ae63
verified

ericaRC commited on

Upload tokenizer
8753533
verified

ericaRC commited on