EOS token on Model Card does not match tokenizer files
#12
by OldSkoolRoolz - opened
The EOS token on model card says it is <|end|> and that is not the token that is defined in the json files in the repo.
The EOS token on model card says it is <|end|> and that is not the token that is defined in the json files in the repo.