nagolinc commited on
Commit
e68ac4e
·
1 Parent(s): df677e9

fix: Correct tokenizer configuration for HF

Browse files
Files changed (1) hide show
  1. hf_model/tokenizer_config.json +2 -3
hf_model/tokenizer_config.json CHANGED
@@ -1,5 +1,4 @@
1
  {
2
  "model_type": "gpt2",
3
- "tokenizer_class": "T5Tokenizer",
4
- "sp_model_file": "spm_chat.model"
5
- }
 
1
  {
2
  "model_type": "gpt2",
3
+ "tokenizer_class": "T5Tokenizer"
4
+ }