eos_token_id is 0 in config.json but 50279 in tokenizer.json
#1
by
NICOLASORZ - opened
eos_token_id is 0 in config.json but 50279 in tokenizer.json. Is there anything wrong?
Great catch, thank you @NICOLASORZ .
I've updated it in config.json, and tokenizer.json already has the correct value (50279).
Let us know if you spot anything else.
dumitrac changed discussion status to
closed