makemore-mlp / tokenizer_config.json

Commit History

revert AutoTokenizer auto_map to array format
2bacd61

George Treacy commited on

fix tokenizer resolution: use auto_map only, remove tokenizer_class
66ab6c3

George Treacy commited on

fix AutoTokenizer auto_map to string not array
9fc7c74

George Treacy commited on

add tokenizer_config.json for AutoTokenizer resolution
f673925

George Treacy commited on