Why is the number of input embeddings 8 greater than the vocabulary size?
#56
by Ali2500 - opened
I noticed that the number of input embeddings (128264) is 8 greater than the number of output embeddings/vocabulary size (128256).
What are the extra 8 embeddings at the end for?
Ali2500 changed discussion title from Why is the number of input embeddings is 8 greater than the vocabulary size? to Why is the number of input embeddings 8 greater than the vocabulary size?