Is the config.json wrong?
#4
by kkyeer - opened
The 4b-pt config.json is much same as 4b-it, and diffrent from 1b-pt version.For example,no vocab_size prop in 4b but can be found in 1b.Is this intended or a mistake?
Same problem here
Hi ,
Thank you for pointing this out! You're correct — the config.json for the 4b-pt and 4b-it versions appears to omit the vocab_size field, while it's present in the 1b-pt version.
This could be intentional if the tokenizer is assumed to be loaded independently and the model config relies on tokenizer metadata. In many recent transformer-based models, the vocab_size is not strictly required in config.json during inference. Thank you.