why vocab size is 32001
#3
by yechenzhi1 - opened
just out of curiosity, why do you increase the vocab size from 32000 to 32001.
because mistral does not have a padding token so we add a [PAD]. you cannot use the eos token as the pad token because in this case the multi-turn conversation does not work probably because the chat template of mistral.