Please add <|eot_id|> as a stop token to the HF config
#12
by omarkilani - opened
Both <|end_of_text|> and <|eot_id|> should be in the config, like they are over at:
Thanks!
-- Groq
omarkilani changed discussion title from Please add <eot_id> as a stop token to the HF config to Please add <|eot_id|> as a stop token to the HF config
yeah, generation does not stop and keeps generating till max tokens reached.
Fixed!
pcuenq changed discussion status to closed
I am still facing this issue. how can i make the response stop?
I fixed the issue by pointing to a tokenizer of another model card which is llama3-70b-Instruct.
@solankibhargav @omarkilani
It seems my model is still generating indefinitely. Where should I include both stop tokens?
@Iionbarista I pointed my tokenizer to https://huggingface.co/v2ray/Llama-3-70B-Instruct and that solved my problem