yunjae-won
/

llama8b_sft

Text Generation

text-generation-inference

Model card Files Files and versions

yunjae-won commited on 20 days ago

Commit

988d0dd

·

verified ·

1 Parent(s): 79d3fa1

Update tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -11,5 +11,5 @@
   "model_max_length": 1000000000000000019884624838656,
   "model_specific_special_tokens": {},
   "pad_token": "<|eot_id|>",
-  "tokenizer_class": "TokenizersBackend"
 }

   "model_max_length": 1000000000000000019884624838656,
   "model_specific_special_tokens": {},
   "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
 }